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ABSTRACT 


This  report  summarizes  research  in  the  development  of 
mathematical  models  of  information  processing  systems  con¬ 
ducted  at  The  University  of  Michigan  Systems  Engineering  Lab¬ 
oratory  during  the  period  from  October,  1967  to  October,  1968 
under  Rome  Air  Development  Center  sponsorship.  Particular 
attention  is  given  to  a  new  approach  to  automata  theory,  the 
use  of  multiple  index  matrices  in  generalized  automata  theory, 
asymptotic  decomposition  of  machines,  reeognizability  of  equa¬ 
tion  sets,  algebraic  isomorphism  invariants  for  transition 
graphs,  iterative  network  realization  of  sequential  machines, 
optimum  sequencing  of  jobs  subject  to  deadlines,  and  the  theory 
of  formal  languages  and  its  impact  on  the  design  and  implemen¬ 
tation  of  programming  languages. 
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INTRODUCTION 

This  report  is  the  final  report  in  a  series  of  yearly  reports  over 
a  four-year  period  concerning  mathematical  models  of  information 
processing  systems  conducted  in  the  Systems  Engineering  Laboratory 
under  the  direction  of  Professor  H.  L.  Garner,  Most  of  the  theories 
relevant  to  computation  relate  to  either  automata  theory  or  the  theory 
of  formal  languages.  This  is  true  for  most  of  the  research  discussed 
in  this  report.  The  exception  is  Section  7.  In  Section  7  the  techniques 
common  to  operations  research  are  used  to  investigate  the  problem  of 
optimum  sequencing. 

In  Section  1  Professor  D.  Muller  outlines  a  new  approach  to 
automata  theory  based  on  the  algebra  of  relations.  This  particular 
approach  provides  basic  and  powerful  conceptual  tools  applicable  to 
the  problem  of  automata  decomposition.  In  the  second  section  Muller 
develops  a  generalization  of  automata  theory  which  is  an  alternative 
to  the  generalized  automata  theory  developed  by  Eilenberg  and  Wright 
based  on  category  theory  and  universal  algebra.  Muller's  generali¬ 
zation  is  based  on  formal  mathematical  notions  used  in  switching  and 
automata  theory  and  provide  a  convenient  way  of  visualizing  the  con¬ 
structions  about  the  systems  which  enable  one  to  anticipate  results. 

In  last  year's  report,  one  section  reported  research  by  Putzolu 
on  the  subject  of  asymptotic  decomposition  of  automata.  His  analysis 
treated  the  case  in  which  state  based  type  of  realization  was  assumed 
and  decomposition  reporting  state  splitting  was  not  permitted.  In  Sec¬ 
tion  3  recent  research  results  of  Putzolu  and  Muller  are  presented 
which  consider  the  case  where  state  splitting  is  allowed. 

Section  4  presents  research  by  C.  R.  Shepard  concerning  gen¬ 
eration  and  recognition  in  formal  languages.  The  research  concerns 
a  generalization  of  the  languages  generated  by  a  left  linear  grammar 
known  as  equational  sets.  Fquationaiity  and  recognizability  can  be  de¬ 
fined  for  any  algebra.  The  conditions  on  an  algebra  such  that  the  defi¬ 
nitions  of  equationality  and  recognizability  coincide  are  given.  If 
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equational  sets  of  an  algebra  are  recognizable,  then  all  word  problem  s 
on  the  algebra  are  solvable.  The  converse  is  false.  These  abstract 
results  are  believed  to  have  some  implication  on  the  design  of  paren¬ 
thesis  free  languages,  and  list  structures. 

In  Section  5  the  results  of  research  on  the  algebraic  isomor¬ 
phism  invariants  for  transition  graphs  by  J.  F.  Meyer  is  presented 
in  complete  but  condensed  form.  The  general  results  of  this  research 
concern  1)  How  the  values  of  the  isomorphism  invariants  relate  to 
invariant  structural  characteristics  and  2)  The  specification  of  special 
classes  of  transition  graphs  for  which  the  isomorphism  invariants 
are  complete.  The  abstract  results  presented  in  this  section  per¬ 
tain  to  the  problems  of  code  design  and  automata  design. 

In  Section  6,  J.  R.  Jump's  research  on  the  iterative  network 
realization  of  sequential  machines  is  summarized.  A  design  algo¬ 
rithm  is  shown  to  exist  for  all  sequential  machines.  A  specific  de¬ 
sign  algorithm  is  specified  and  bound  on  the  complexity  of  the  iter¬ 
ative  realization  is  given  for  different  classes  of  sequential  machines. 
This  study  is  relevant  to  the  application  of  £si  technology. 

In  Section  7,  research  by  E.  L.  Lawler  and  M.  Moore  concern¬ 
ing  the  sequence  of  tasks  subject  to  deadline  constraints  is  presented. 
The  class  of  sequences  considered  are  those  for  which  the  consisten¬ 
cy  principle  hold.  For  this  class  there  exists  a  linear  ordering  of 
tasks  which  determines  an  optimal  schedule.  This  principle  is  ex¬ 
ploited  to  produce  solution  methods  more  efficiently  than  previous 
methods. 

In  Section  8,  L.  Liu  presents  the  structure  of  formal  languages 
and  relates  this  structure  to  some  of  the  problems  involved  in  com¬ 
puter  languages.  Computer  languages  do  not  exactly  fit  the  existing 
classification  structure  and  directions  for  future  research  are  indi¬ 
cated. 


Section  1 


A  New  Approach  to  Automata  Theory 

This  section  presents  a  new  approach  to  automata  theory  based 
on  an  aigebra  of  relations  which  is  applicable  to  automata  theory  in 
much  the  same  way  Boolean  algebra  is  applicable  to  switching  t>  ary. 

One  of  the  purposes  of  this  approach  is  didactic.  Previous  ex¬ 
perience  has  shown  that  this  method  of  teaching  automata  theory  is 
useful  with  students  who  have  had  little  contact  with  the  conventional 
methods  and  that  it  provides  them  with  a  more  powerful  way  of  think¬ 
ing  about  the  subject. 

1.1  An  Algebra  of  Relations 

1. 1. 1  Let  Q  and  Q'  be  two  nonempty  sets,  and  let  Q  x  Q'  repre¬ 
sent  the  Cartesian  product  or  set  consisting  of  all  ordered  pairs 

(q,  q’),  where  q  is  an  element  of  Q  and  q'  is  an  element  of  Q’.  We 
may  use  the  formal  mathematical  notation: 

QXQ'  =  | (q,  q’)  I  q  €  Q,  q' t  Q'}. 

A  subset  R  of  such  a  Cartesian  product  Q  x  Q’  is  a  binary  rela- 
tion.  When  defining  such  a  binary  relation  we  must  give  all  three  sets 
Q,  Q'  and  R.  We  shall  call  Q  and  Q’  the  input  and  output  sets  respec¬ 
tively  and  R  the  set  of  pairs.  Thus,  we  regard  two  relations  as  equal 
only  if  they  have  the  same  input  and  output  sets  as  well  as  the  same 
set  of  pairs. 
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There  are  three  other  convenient  ways  that  we  may  think  about 
binary  relations.  First,  if  we  are  given  a  relation  R  C  Q  x  Q\  then 
we  may  imagine  a  Boolean  matrix  whose  rows  are  labelled  by  the  ele¬ 
ments  q  of  Q  and  whose  columns  are  labelled  by  the  elements  q'  cf  Q'. 
The  matrix  element  in  the  row  q  and  column  q'  is  made  1  if  and  only  if 
(q,  q')  is  in  R.  Otherwise  it  is  made  0.  It  is  clear  that  for  every  bi¬ 
nary  relation  we  may  imagine  such  a  matrix  and  that  conversely  for 
every  Boolean  matrix  we  have  a  corresponding  binary  relation. 

Second,  we  may  think  of  the  relation  RCQxQ'as  representing 
a  process.  If  we  are  given  some  member  q  of  the  input  set  Q  then  the 
process  yields  an  element  q’  of  the  output  set  Q’  such  that  (q,  q')  is  in 
R.  If  no  such  q'  exists,  then  the  process  must  be  assumed  to  produce 
no  output.  On  the  other  hand,  if  several  such  elements  q'  exist,  then 
we  suppose  that  the  process  makes  an  arbitrary  and  unpredictable 
choice  of  one  of  them.  Since  a  process  must  be  described  more  care¬ 
fully  and  exactly  than  has  been  done  here  if  it  is  to  be  useful  mathe¬ 
matically.  we  shall  simply  regard  this  model  as  a  convenient  intuitive 
way  of  thinking  about  relations. 

Third,  we  may  think  of  a  relation  as  being  described  by  a  directed 
linear  graph.  This  model  is  particularly  useful  when  the  relation  is 
square,  ihat  is,  when  Q  =  Q’.  The  nodes  of  the  graph  are  \hen  just 
the  elements  of  Q  and  for  each  pair  (q,  q’)  in  R.  we  draw  a  line  from  q 
to  q’  with  an  ariow  pointing  toward  q’. 
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It  is  also  possible  to  represent  relations  which  are  not  square 
by  means  of  linear  graphs.  Suppose  that  R  c_  Q  x  Q'  is  not  square, 
i.  e. ,  Q  4  Q\  Then  we  may  form  a  corresponding  square  relation 
R  c  (Q  u  Q')  x  (Q  u  Q').  In  this  case  we  use  the  union  Q  u  Q'  as  both 
the  input  and  output  sets.  The  directed  linear  graph  corresponding  to 
this  square  relation  ma>  also  be  used  to  describe  the  original  relation 
provided  the  input  and  output  sets  are  specified. 

Binary  relations  are  sets  and  therefore  we  may  use  Boolean  alge¬ 
bra  when  dealing  with  them.  Thus,  if  R  C  Q  x  Q'  and  S  cQxQ'  are 
two  relations  with  the  same  input  and  output  sets,  then  we  may  form 
their  set  theoretical  union  R  u  S  C  Q  x  Q'  and  intersection  R  n  S  C 
Q  x  Q\  The  union  and  intersection  are  both  taken  as  relations  having 
the  same  input  and  output  sets  as  R  and  S.  Furthermore,  any  relation 
R  C  Q  x  Q’  may  be  complemented  giving  R  =  Q  x  Q'  -  R  which  consists 
of  all  pairs  (q.  q')  in  Q  x  Q'  but  not  in  R.  We  also  note  that  relations 
with  given  input  and  output  sets  are  partially  ordered  by  set  inclusion, 
written  R  C  S. 

Relations  are  quantities  with  more  mathematical  structure  than 
sets,  and  therefore  we  may  define  other  important  operations  on  re¬ 
lations.  If  R  C  Qx  Q’  and  S  C  Q'  x  Q”  are  two  relations  such  that 
the  output  set  Q'  of  R  is  the  same  as  the  input  set  of  6.  then  we  may 
define  the  Pierce  product  R  0  S  of  R  by  S.  We  take  R  5  S  C_  Q  x  Q" 
as  the  relation  consisting  of  ail  pairs  (q,  q")  such  that  for  some  q’  in 
Q'  we  have  (q.  q ’ )  in  R  and  (q\  q”)  in  S.  We  may  use  the  formal 
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mathematical  notation: 

R  0  S  =  {(q,q")l  B'q’  e  Q!,  (q,q’)  e  R,  and  (q',  q")  e  S}. 

If  R  and  S  are  regarded  as  corresponding  to  Boolean  matrices, 
then  R  0  S  corresponds  to  a  form  of  Boolean  matrix  product  in  which 
the  summation  which  is  used  in  the  conventional  definition  of  matrix 
product  is  replaced  by  the  Boolean  operation  "or".  Similarly,  if  R 
and  S  are  thought  of  as  representing  processes,  then  R  °  S  represents 
a  combined  process  in  which  the  output  of  R  is  used  as  the  input  for  S. 

We  further  define  the  inverse  or  transpose  R~ 1  of  the  relation 
RC  Qx  Q'  as  consisting  of  all  pairs  (q',  q)  such  that  (q,  q')  is  in  R. 
Thus,  R_1  is  a  subset  of  Q'  x  Q  so  that  the  input  and  output  sets  are 
interchanged  when  taking  the  transpose.  In  the  case  of  the  linear 
graph  model,  the  transpose  is  formed  by  reversing  the  directions 
of  the  arrows  of  the  graph. 

We  will  now  show  a  precise  correspondence  between  identities 
on  Boolean  matrices  and  identities  on  matrices  of  integers. 

Let  Z  be  the  set  of  nonnegative  integers  and  define  a  mapping 
(3  from  Z  to  the  3et  {0,  1}  by  the  rule  that  (3  maps  0  to  0  and  /3  maps 
l  to  1  if  l  >  0.  If  M  and  N  are  matrices  whose  elements  are  nonneg¬ 
ative  integers,  then  let  M'  and  N'  be  the  corre  ponding  Boolean  ma¬ 
trices  obtained  by  replacing  the  elements  of  M  and  N  respectively 
with  their  images  under  the  mapping  p.  We  may  easily  derive  the  fol¬ 
lowing  rules  which  allow  us  to  regard  ii  as  a  kind  of  homomorphism. 


7 


1.  (M+N)'  =  M'  UN' 

2.  (M-N)'  =  M!  °  N1 

3.  (M  0  N)’  =  M'  (IN' 

4.  (MT)'  =  (M’)_1 

5.  If  M  <N,  then  M'  C  N'. 

Here,  the  product  MON  means  elementwise  multiplication  of  the 

T 

two  matrices  and  M  represents  the  transpose.  Yet  a  sixth  rule 
might  be  written  for  multiplication  of  a  matrix  by  a  scalar. 

The  preservation  of  the  various  operations  under  the  homomor¬ 
phism  /3  allows  us  to  transform  any  identity  involving  matrices  of  non- 
negative  integers  and  these  operations  into  an  identity  for  Boolean  ma¬ 
trices. 

Not  all  operations  on  matrices  of  nonnegative  integers  are  pre¬ 
served  under  (i.  For  example,  the  matrL*  difference  M-N  does  not 
map  uniformly,  and  identities  involving  the  difference  do  not  generally 
correspond  to  identities  on  Boolean  matrices.  Furthermore,  since 
the  mapping  is  not  one-to-one  there  may  be  identities  which  are  valid 
for  Boolean  matrices  but  which  do  not  hold  for  matrices  of  nonnegative 
integers. 

1.  1.  2  In  our  study  of  the  properties  of  relations  we  shall  need 
to  consider  relations  with  certain  special  properties,  and  we  there¬ 
fore  proceed  to  define  terms  for  some  of  these  properties.  Let 
R  C  Q  x  Q‘  be  a  relation,  then: 
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1.  K  is  determinate  iff  for  each  q  e  Q  there  1°  at  most  one 
q’  e  Q'  such  that  (q,  q')  is  in  R. 

2.  R  is  productive  iff  for  each  q  e  Q  there  is  at  least  one 
q'  e  Q'  such  that  (q,  q')  is  in  R. 

3.  R  is  a  single  valued  iff  it  is  both  determinate  and  produc¬ 
tive.  It  is  then  called  a  mapping  or  a  function  from  Q  to  Q\ 

4.  R  is  surjective  or  onto  iff  for  each  q'  e  Q'  there  is  at 
least  one  q  e  Q  such  that  (q,  q’)  is  in  R. 

5.  R  is  injective  iff  it  is  single  valued  and  R  1  is  determinate. 

6.  R  is  one-to-one  iff  R  and  K  1  are  both  single  valued. 

Using  the  matrix  model,  we  see  that  if  R  is  determinate,  then 
each  row  of  the  corresponding  matrix  may  have  at  most  one  element 
which  is  1.  Similarly,  if  R  is  productive  then  each  row  contains  at 
least  one  1. 

In  our  model  of  R  as  a  process  we  see  that  if  R  is  determinate 
then  the  result  q’  if  it  exists,  is  uniquely  determined  by  the  input  q. 
Similarly,  if  R  is  productive  then  some  output  q’  must  occur  for  ev¬ 
ery  input. 

We  shall  write  <f>  to  denote  an  empty  relation.  The  set  <p  is  re¬ 
garded  as  a  subset  of  Qx  Q'  but  containing  no  pairs.  This  special 
relation  has  the  prope  rties  of  a  zero  with  respect  to  Pierce  product. 
That  is,  R  °  <f>  =  <f>  o  R  ~  <p  whenever  such  multiplication  is  possible 
with  a  relation  R.  The  Boolean  matrix  corresponding  to  <j>  is  a  matrix 


whose  elements  are  all  0. 
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If  Q  is  a  set,  then  define  <  Q  as  the  set  of  all  pairs  (1,  q),  where 
q  is  in  Q  and  1  is  the  element  of  a  singleton  set  1.  Thus,  <  Q  may  be 
regarded  as  a  row  vector  whose  elements  are  all  1.  The  transpose 
of  <  Q  we  shall  write  as  Q  >  and  it  consists  of  all  pairs  (q,  I)  where  q 
is  in  Q.  We  note  that  Q  >  is  a  column  vector.  A  similar  notation  may 
be  used  for  subsets  of  Q.  Thus,  if  K  C  Q  then  <  K  and  K  >  represent 
row  and  column  vectors  respectively  whose  elements  are  1  precisely 
for  members  of  K. 

Row  and  column  vectors  rnay  be  used  with  the  Pierce  product 
notation.  In  particular,  if  R  C  Q  x  Q*  is  any  relation,  then  <Q°R 
is  called  the  range  of  R  and  R  »  Q’  >  is  called  the  domain  of  R.  We 
note  also  that  Q  >  °  <  Q'  =  Q  x  Q’,  providing  a  convenient  abbreviation. 
1.  1.  3  We  now  direct  our  attention  to  the  case  of  square  relations. 
A  particular  square  relation  QxQof  some  importance  is  called 
the  identity  on  Q.  It  consists  of  just  those  pairs  (q,  q')  for  which  q  =  q’. 
Thus,  Iq  =  [(q,  q)  I  q  €  Q].  The  Boolean  matrix  corresponding  to  1^ 
consists  of  l’s  along  the  diagonal  and  0’s  everywhere  else. 

Let  RCQxQ  be  a  square  relation,  then: 

7.  R  is  symmetric  iff  whenever  (q,  q’)  is  in  R,  then  (q\  q) 
is  also  in  R. 

8.  R  is  antisymmetric  iff  whenever  (q,  q’)  is  in  R  and  q  ^  q’, 
then  (q\  q)  is  not  in  R. 

9.  R  is  reflexive  iff  for  each  q  e  Q,  the  pair  (q,q)  is  in  R. 
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10.  R  is  irreflexive  iff  for  each  q  e  Q,  the  pair  (q,  q)  is  not 
in  R. 

11.  R  is  weakly  reflexive  iff  whenever  (q,  q’)  is  in  R,  then 
(q',q')  is  in  R. 

12.  R  is  dually  weakly  reflexive  iff  whenever  (q,  q’)  is  in  R, 
then  (q,  q)  is  m  R. 

13.  R  is  transitive  iff  whenever  (q,  q')  and  (qT,  q")  are  both 
in  R,  then  (q,  q")  is  in  R. 

14.  R  is  an  equivalence  relation  iff  it  is  reflexive,  symmetric, 
and  transitive. 

15.  R  is  a  partial  order  iff  it  is  reflexive,  antisymmetric, 
and  transitive. 

16.  R  is  a  total  order  iff  it  is  a  partial  order  and  every  pair 
(q,  q')  in  Q  x  Q  is  either  in  R  or  in  R  . 

Using  our  matrix  model,  we  see  that  if  R  is  weakly  reflexive 
then  any  column  in  the  corresponding  matrix  which  is  not  all  0  has 
a  diagonal  element  which  is  1.  In  the  graph  corresponding  to  a  weakly 
reflexive  relation,  any  node  having  an  arrow  pointing  toward  it  must 
also  have  a  self  loop. 

The  terms  "antisymmetric"  and  "irreflexive"  are  subject  to 
misinterpretation.  There  are  relations,  such  as  1^,  which  are  both 
symmetric  and  antisymmetric,  so  that  antisymmetry  is  not  the  denial 
of  symmetry.  Also,  some  relations  are  neither  symmetric  nor  anti¬ 
symmetric  and  some  relations  are  neither  reflexive  nor  irreflexive. 
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Many  of  the  special  properties  we  have  defined  have  formal  in¬ 
terpretations.  Examples  of  such  interpretations  are  contained  in  the 
following  list  in  which  it  is  assumed  that  R  has  the  property  in  ques¬ 


tion. 

1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 
9. 


Determinate:  R  0  R  Iq»  • 

Productive:  R  °  R  ^  D  In,  or  alternatively 
R»Q’>  =  Q>. 

Symmetric:  R  =  R  \ 


Antisymmetric:  R  0  R  c  Iq. 

Reflexive:  R  D  JL. 

—  y 

Irreflexive:  R  ?  1^,  or  alternatively  R  n  L  =  f 
Weakly  reflexive:  R  o  (R  n  Iq)  =  R. 

Dually  weakly  reflexive:  (R  n  Iq)  °  R  =  R. 
Transitive:  R  0  R  c  R. 


1.  1.  4  If  R  £L  Q  x  Q  is  any  square  relation,  then  although  it  may 
not  be  reflexive,  we  may  obtain  a  reflexive  relation  from  it  by  the  sim¬ 
ple  device  <  f  forming  R  ul  .  This  new  relation  K  u  I0  will  be  called 
the  reflexive  extension  of  R  and  may  be  constructed  using  our  matrix 
model  by  simply  replacing  each  diagonal  element  of  R  by  1. 


Similarly,  R  may  not  be  symmetric,  but  we  may  use  R  to  form 
the  relation  R  u  R'1  which  is  symmetric.  It  is  also  true  that  R  n  R  * 


is  symmetric,  but  we  shall  define  the  symmetric  extension  of  R  to  be 
R  u  R  *  rather  than  R  n  R  \ 
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Finally,  we  define  R  u  R"^  u  Las  the  symmetric  and  reflexive 
extension  of  R.  We  note  that  it  is  both  the  symmetric  extension  of  the 
reflexive  extension  and  the  reflexive  extension  of  the  symmetric  ex¬ 
tension.  Clearly,  R  u  R-*  u  I  is  both  symmetric  and  reflexive. 

(ef 

Somewhat  less  obvious  is  how  one  might  construct  a  transitive 
extension  of  a  square  relation  R.  This  extension,  however,  will  be 
very  important  in  our  later  analysis  and  for  this  reason  we  designate 
it  by  the  special  symbol  R+.  We  define  R+  as  the  relation  consisting 
of  all  pairs  (q,  q’)  such  that  for  some  finite  sequence  q^,  qg,  . . . ,  qfl 
of  elements  of  Q  we  have  q  =  qj  and  qn  =  q'  while  each  consecutive 
pair  (q.,  q.+1)  of  terms  in  the  sequence  is  a  member  of  R.  We  do  not 
require  that  the  terms  of  the  sequence  all  be  distinct,  but  we  do  as¬ 
sume  that  the  number  n  of  terms  is  at  least  2. 

A  formal  representation  of  the  above  definition  is: 

R+  =  RjRoRuRoRoR...  . 

In  the  definition  ci  'he  transitive  extension  if  we  had  allowed  n  to 
be  1  as  well  as  larger  incogs* s,  then  we  obtain  a  relation  which  will 
be  designated  by  R*.  It  is  called  the  reflexive  and  transitive  exten¬ 
sion  of  R.  One  may  show  quite  easily  that  R*  =  R+  u  1^  and  that 
R+  =  R  °  R*.  Thus,  R+  and  R*  are  transitive  but  R+  may  not  be  re¬ 
flexive  while  R*  is  always  reflexive. 

We  define  R  C  Q  x  Q  to  be  acyclic  iff  R*  is  irreflexive.  Using 
the  model  of  a  directed  linear  graph  we  have  a  convenient  interpreta¬ 
tion  for  this  property.  We  note  from  our  definition  of  R+  that  it  consists 


•rv)" 
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of  all  pairs  (q,  q’)  which  are  joined  by  a  chain  of  sequence  qA,  qg,  ... 
qn  which  starts  at  q  =  q^  and  ends  at  qn  =  q'.  Thus,  if  R  is  irreflex- 
ive  we  can  never  have  q  =  q\  In  other  words,  there  can  be  no  loops 
or  cycles  in  the  graph  which  permit  one  to  pass  from  some  point  back 
to  itself  following  the  direction  of  the  arrows. 

A  familiar  example  of  an  acyclic  relation  is  provided  by  the  con¬ 
cept  of  a  combinational  circuit.  In  such  a  circuit  we  let  Q  represent 
the  set  of  nodes  of  the  circuit  and  place  (q,  q')  in  R  whenever  there  is 
a  switching  element  having  q  for  an  input  and  q*  for  an  output.  Thus, 
if  the  circuit  is  combinational  there  is  no  feedback  and  hence  no  cycles 
of  the  type  described.  This  is  equivalent  to  saying  that  R  is  acyclic. 

1.  1.  5  If  R  C_  Q  x  Q  is  any  square  relation,  then  we  define  the 
trace  of  R  which  is  written  Tr(R)  to  be  1  if  (q,  q)  is  in  R  for  some 
q  €  Q  and  to  be  0  otherwise.  In  terms  of  our  matrix  model  we  form 
Tr(R)  by  taking  the  union  of  the  diagonal  elements  of  the  matrix  corres 
ponding  to  R.  Also,  in  the  linear  graph  model  we  no.e  that  Tr(R)  =  1 
if  the  graph  corresponding  to  R  has  any  self  loops  and  Tr(R)  =  0  other¬ 
wise. 

If  we  regard  1  as  standing  for  truth  and  0  as  standing  for  false¬ 
hood,  then  Tr(R)  represents  the  preposition  (3  q  e  Q,  (q,  q)  e  R). 

In  the  case  of  matrices  whose  elements  are  integers,  it  is  cus¬ 
tomary  to  define  the  trace  of  a  matrix  as  the  sum  of  its  diagonal  ele¬ 
ments.  Since  our  mapping  /J:  Z  -  {0.  1}  is  a  homomorphism  if  sum¬ 
mation  becomes  union,  we  see  that  the  trace  operation  is  preserved 
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under  this  mapping.  That  is  to  say,  the  image  of  Tr(M)  under  /3  is 
just  Tr(M')  if  M'  is  the  image  of  M. 

1 

1.  1.  6  It  is  easily  shown  that  if  R  is  any  relation,  then  R  °  R 
is  square,  symmetric,  and  weakly  reflexive.  We  therefore  may  won¬ 
der  if  whenever  we  have  a  relation  S  C  Q  x  Q  which  is  square,  sym¬ 
metric,  and  weakly  reflexive,  whether  it  can  be  written  in  the  form 
R  °  R_1  =  S  for  some  relation  R.  This  question  will  be  answered  in 
the  affirmative,  but  some  proof  is  required. 

Let  S  C  Q  x  Q  be  given  as  above  and  define  a  family  ~K  of  subsets 
K  of  Q  by  placing  K  in  %  iff  it  satisfies  the  following  property  for 
every  q  e  Q.  The  element  q  is  in  K  iff  (q,  q’)  is  in  S  for  each  q’  e  K. 

A  formal  definition  of  K  may  be  written  as  follows: 

%  =  {K  C  Q  I  V  q  €  Q,  [q  €  K  <=>  Vq’e  K,  (q,  q')  e  S]} . 

Although  this  definition  of  the  amily  H  is  complicated  we  gain  some 
insight  into  its  nature  and  structure  by  developing  some  of  its  prop¬ 
erties. 

Define  a  subset  J  of  Q  to  be  complete  iff  every  pair  (q,  q’l  of 
not  necessarily  distinct  elements  of  J  is  also  a  pair  in  S.  A  maximal 
complete  set  is  one  which  is  not  properly  contained  in  any  other  com¬ 
plete  set. 

Lemma  1_ 

*A  is  the  family  of  all  maximal  complete  sets. 

Proof:  If  K  is  in  K  and  q  is  in  K,  then  (q.  q')  f  S  for  each  q' t  K. 

Hence  every  pair  (q.  q')  of  not  necessarily  distinct  elements  of  K 
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must  be  a  pair  in  S,  so  K  is  complete.  In  fact,  we  see  that  this  prop¬ 
erty  expresses  the  "only  if"  part  of  our  defining  condition  for  K  . 

Next,  consider  a  complete  set  J  which  is  not  maximal.  There  must 
be  some  other  complete  set  J*  such  that  J  C  J'.  Therefore,  there  is 
at  least  one  element  q  e  J'  such  that  q  is  not  in  J.  Since  J’  is  com¬ 
plete,  we  see  that  (q,q')  is  inS  whenever  q'  is  in  J.  Thus,  J  cannot 
be  in  %  and  every  set  K  e  A  must  be  a  maximal  complete  set.  We 
see  that  this  latter  property  expresses  the  "if"  part  of  our  defining 
condition  for  H .  Hence,  the  defining  condition  is  satisfied  by  every 
maximal  complete  set,  thus  proving  the  lemma. 

Lemma  2 

Whenever  (q,  q’)  is  a  pair  in  S,  there  is  some  K«  ^ 

such  that  both  q  and  q'  are  in  K. 

Proof:  If  q  =  q',  then  the  singleton  set  {q}  is  complete.  Otherwise, 
if  q  ^  q',  then  the  two  element  set  {q,  q'}  is  complete  because  (q’,  q), 

(q,  q)  and  (q\  q')  must  all  be  in  S  as  a  result  of  the  fact  that  S  is  sym¬ 
metric  and  weakly  reflexive.  Thus,  q  and  q'  are  both  in  some  com¬ 
plete  set  Cp  Let  be  the  family  of  all  complete  sets  C  which  in¬ 
clude  C^.  Then  C  is  partially  ordered  under  set  inclusion  and  ful¬ 
fills  the  condition  of  Zorn's  lemma,  namely  that  every  chain  in 
have  an  upper  bound.  Hence,  we  conclude  from  Zorn's  lemma  that  < 
has  a  maximal  element  K.  This  element  is  also  maximal  in  the  family 
of  all  complete  sets  since  any  set  including  K  must  also  include  C^. 
Hence  K  is  in  ^  by  lemma  1.  and  we  have  completed  our  proof. 
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Remark  1 

There  may  be  other  families  of  maximal  complete  sets 
which  have  the  property  expressed  in  lemma  2. 

Proof;  We  give  a  simple  example  of  such  a  case.  Let  Q  be  a  six 
element  set  { 1,  2, 3,  4,  5, 6}  and  let  S  contain  the  pairs  (1,  2),  (2,  3), 

(3,  1),  (2,  4),  (3,  4),  (1,  5),  (3,  5),  (1, 6),  (2,  6)  as  well  as  all  other  pairs 
required  to  satisfy  symmetry  and  reflexivity.  Then,  the  graph  has 
the  following  form: 


o 

The  family  of  three  maximal  complete  sets  { 1,  2,  6},  {2,  3,  4} ,  { 1,  3,  5} 
satisfies  the  condition  of  lemma  2  but  it  does  not  contain  the  maximal 
complete  set  { 1,  2,  3}.  Thus,  the  remark  is  proved. 

Continuing  with  our  original  development,  we  introduce  a  set 


W,  called  the  normal  set  which  is  in  one-to-one  correspondence  with 
the  family  /V  .  We  write  w~  K  to  indicate  that  the  element  w  e  V/ 
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corresponds  to  the  set  K  e  H  •  This  set  W  is  to  be  used  as  a  nota- 
tional  convenience  in  describing  properties  of  the  relation  S.  When 
it  is  necessary  to  specify  S  we  shall  use  subscript  notation  as  W  and 

S 

K  . 

s 

J  J 

Define  the  relation  SQ  c  Q  x  W  by  the  rule  that  (q,  w)  is  in  Sa 
iff  q  e  K,  where  w  ~  K.  Wc  shall  write  s'd  as  an  abbreviation  for 


Theorem  1 

If  S  C_  Q  x  Q  is  symmetric  and  weakly  reflexive, 
then  6  =  Sd  °  S  d. 

Proof:  If  (q,  q')  is  in  S  then  by  lemma  2  there  is  a  set  K  of  %  con¬ 
taining  both  q  and  q\  Pick  w  e  W  which  corresponds  to  this  set. 
Since  (q,  w)  is  in  Sd  and  (w,  q')  is  in  S~d,  we  see  that  (q,  q?)  is  in 
Sd  o  S"d  Hence,  S  C  Sd  •  S_d 

Next,  suppose  that  (q,  q')  is  in  Sd  °  S  d.  Then,  there  must  be 

ci 

some  element  w  e  W  such  that  (q,  w)  is  in  S  and  (w,  q')  is  in  S  . 

Pick  K  €  %  corresponding  to  this  w.  Since  both  q  and  q'  are  in  K 
we  have  (q,  q’)  f.  S  by  lemma  1,  and  the  theorem  is  proved. 

Remark  2 

If  S  is  transitive  then  Sd  is  determinate  and  the  family 
'h  is  pairwise  disjoint. 

Proof:  Let  K  and  K'  be  two  not  necessarily  distinct  elements  of 
and  assume  K  ■  '  K'  ^  •}>.  If  q ^  is  in  K  >  K’  then  (q,  q j,)  is  in  S  for  all 
q  €  K  since  K  is  complete  and  similarly  (q^,  q’}  is  in  S  for  all  q'  c  K’. 
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Since  S  is  transitive,  we  see  that  (q,  q')  must  always  be  in  S  whenever 

q  is  in  K  and  q'  is  in  K\  But,  S  is  symmetric  and  weakly  reflexive, 

and  therefore  K  u  K'  is  complete.  However,  K  and  K'  are  maximal 

complete  sets  so  therefore  K  =  K\  Therefore,  if  K  4  K',  we  have 

K  n  K'  =  <p  and  is  pairwise  disjoint. 

d 

To  show  that  Su  is  determinate  we  consider  any  element  q  c  Q. 

If  (q,  w)  is  in  S  ,  then  q  is  in  the  set  K  e  which  corresponds  to 

w  €  W.  But,  q  can  be  in  no  other  set  K'  of  CA'  so  there  is  no  other 

d  d 

element  w'  of  W  such  that  (q,  w')  is  in  S  .  Hence  Su  is  determinate 
and  the  remark  is  proved. 

Remark  3 

If  S  is  reflexive  then  S  is  productive  and  ft?  is  a  cover 

of  Q  in  the  sense  that  l  J  K  =  Q. 

Ke  X 

Proof:  Let  q  be  any  element  of  Q.  Then  (q,  q)  is  in  S  since  S  is  re¬ 
flexive  so  che  singleton  set  -Jq]  is  complete.  By  lemma  2,  there  is 

some  K  €  Y  such  that  q  e  K,  and  hence  is  a  cover.  Also  if 

d  d 

w  ~  K,  then  (q,  w)  is  in  S  so  S  is  productive. 

Remark  4 

d 

If  S  is  an  equivalence  relation  then  S  is  single 
valued  and  X  is  a  partition  of  Q. 

Proof:  The  conditions  of  both  remarks  2  and  3  are  satisfied  if  S  is 
an  equivalence  relation  since  it  is  then  both  i*flexive  and  transitive. 

.«  .  d 

Hence  Z,  is  a  partition,  i.  e. ,  a  pairwise  disjoint  cover.  Also  S 
is  determinate  and  productive,  thus  single  valued. 
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We  note  that  the  sets  of  are  called  equivalence  sets  when 
S  is  an  equivalence  relation. 

1.  1.  7  Define  the  relation  R  C  Q  x  Q'  to  be  finitary  iff  there  is 

a  finite  set  Q"  which  admits  two  relations  G  c  Qx  Q"  and  H  C_  Q"  x  Q' 

such  that  R  =  G  0  H.  We  note  that  any  finite  relation  R  must  be  fini- 

tary  since  R  =  R  «  L,. 

w 

Theorem  2 

If  S  C  Q  x  Q  is  symmetric  and  weakly  reflexive, 
then  S  is  finitary  iff  the  corresponding  family 
is  finite. 

Proof:  If  is  finite  then  W  is  finite  and  S  =  °  S-C*,  where 

j  j 

SaC  QXW  and  S  a  C  W x  Q  so  S  if  finitary. 

Next,  suppose  that  S  is  finitary  and  hence  may  be  written  in  the 
form  S  =  G  °  H  where  G  cQxQ”  and  H  C  Q"  X  Q  for  finite  Q".  De¬ 
fine  an  equivalence  relation  E  C  Q  x  Q  by  the  rule  that  (q^,  qg)  is  in 
E  iff  for  ali  q"  €  Q"  we  have  (q^,  q")  e  G  <=>  (qg.q")  e  G.  It  is  triv¬ 
ial  to  show  that  E  is  an  equivalence  relation.  Also,  the  number  of 
equivalence  sets  is  finite  since  if  Q"  is  of  cardinality  n  the  number  of 
equivalence  sets  can  be  no  more  than  2n.  We  further  see  that  if 
(q p  q2)  is  in  E,  then  for  all  q  €  Q  we  have  (q^  q)  €  S  <— >  (q2-  q)  e  S. 
Thus,  qj  and  q2  will  always  be  in  exactly  the  same  sets  K  of 
since  the  rule  for  inclusion  or  exclusion  applies  equally  to  both  and  S  is 
weakly  reflexive  so  that  inclusion  of  one  will  not  cause  exclusion  of  the 
other.  This  means  that  each  K  of  i  is  a  union  of  E  equivalence  sets. 
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Thus,  /\.  is  finite  since  the  cardinality  of  /i  can  be  no  greater  than 
the  number  of  possible  unions  of  E  equivalence  sets  which  may  be 
formed. 

Define  the  weight  of  a  finitary  relation  R  c_  Q  x  Q’  to  be  the 

minimum  cardinality  of  any  set  Q"  such  that  R  =  G  0  H  for  some 

2 

GC  Qx  Q"  and  H  C  Q"  x  Q'.  Further,  let  R  be  an  aobreviation 
for  R  °  R,  and  in  general,  Rn  for  the  n-fold  Pierce  product  of  R. 
Theorem  3 

If  R  C_  Q  x  Q  is  finitary  and  of  weight  n,  then 
R+  =  RU  R2  U...  uRn. 

Proof:  From  the  definition  of  R+,  we  know  that  R+  dRu  u  . . , 

u  Rn.  Hence,  to  show  equality  it  is  only  necessary  to  prove  that 

any  pair  (q,  q')  e  R+  is  also  inR  u  R2  u  ...  u  Rn  If  (q,  q  ’)  is  in  R+, 

then  (q,  qr)  is  in  Rrn  for  some  integer  m,  and  we  may  take  m  >  n  for 

otherwise  the  conclusion  is  obvious.  Write  R  =  G»  H  as  above,  where 

G  C  Q  x  Q"  and  HCQ"x  Q,  and  Q"  has  cardinality  n.  Then,  Rm  = 

m  1 

G  0  H  o  . . .  og°H  =  G0(H0  G)  °  H,  where  H«GCQ"x  Q" 

o 

and  we  adopt  the  convention  that  (H  °  G)  =  Iq„.  Since  (q,  q’)  is  in 

Rm,  there  must  be  a  pair  (p,  p’)  €  {H  o  G)m  1  such  that  (q,  p)  is  in  G 

1 

and  (p\  q’)  is  in  H.  However,  if  (p,  p’)  is  in  (H°G)  ,  there  is  a 

sequence  p  =  p^,  Pg,  ...»  Pm  =  P'  such  that  each  consecutive  pair 
(Pi>  Pi+^)  is  in  H  0  G.  But,  Q"  contains  only  n  elements  and  m  >  n 
so  the  terms  in  the  sequence  cannot  ail  be  distinct.  Wc  can  therefore 
contract  the  sequence,  for  if  p.  =  p^,  then  p  =  pj,  Pg,  . . . ,  p^,  p^+i, 
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. . . ,  p  =  p'  satisfies  the  requir  ed  condition  on  consecutive  pairs. 

ni 1 

Hence  there  is  some  integer  m'  <  n  such  that  (p,  p’)  is  in  (H  o  G) 
and  therefore  (q,  q')  is  in  G  0  (H  »  G)m  1  o.H  =  Rm  .  This  completes 
the  proof  of  the  theorem. 

Theorem  4 

If  R  C  Q  x  Q”  is  finitary,  then  R  =  Q  x  Q'  -  R  is 
finitary. 

Proof:  Let  us  write  R  =  G  °  H,  where  G  C  Q  x  Q"  and  H  C  Q"  x  Q\ 
with  Q"  finite.  Let  :7  (QM)  be  the  family  of  all  subsets  of  Q".  Con¬ 
struct  a  relation  G'  C  Q  x  ^'(Q")  by  placing  (q,  P)  in  G'  iff  P  con¬ 
sists  of  just  those  elements  q"  e  Q"  such  that  (q,  q")  is  in  G.  Evi¬ 
dently,  G'  is  single  valued.  Alsoj  construct  a  relation  H'  C 
x  Q'  by  placing  (P,  q’)  in  H’  iff  (q",  q')  is  not  in  H  whenever  q"  is  in 
P.  Consider  the  product  G’  °  H'.  If  (q,  q')  is  in  G’  °  H',  then  for  the 
unique  P  associated  with  q,  we  have  (P,  q')  not  in  H  so  (q,  q')  is  not  in 
G  °  H  =  R.  On  the  other  hand,  if  (q,  q')  is  not  in  G’  »  H',  then  for  the 
unique  P  associated  with  q,  we  have  (P,  q')  not  in  H\  Thus,  there  is 
some  q"  e  P  such  that  (q”,q  '  is  in  H.  Hence,  (q,  q’)  is  in  G  °  H  =  R. 
Therefore,  G’  ®  H’  =  R  and  since  Q"  is  finite,  t.P( Q")  is  also  finite, 
showing  that  R  is  finitary. 
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1.  2  Relational  Systems 

1.  2.  1  To  describe  a  relational  system  S,  one  must  specify: 

(1)  An  alphabet  A  which  is  a  finite  set  of  letters  a  which  are  sym¬ 
bols  used  for  indexing  and  labelling  expressions. 

(2)  A  carrier  set  Q  consisting  of  states  or  elements  q  oi  the  system. 

(3)  A  mapping  F  from  A  to  the  family  of  square  relations  on  Q. 

The  image  of  a  €  A  under  F  will  be  written  F  and  called  the 
set  of  transitions  under  a.  Thus,  F„  CQxQ. 

We  may  think  of  a  relational  system  S  as  being  equal  to  the  triple 
(A,  Q,  F).  It  may  be  represented  by  a  labelled  graph  wnose  nodes  cor¬ 
respond  to  the  states  q  of  Q.  Between  every  pair  (q,  q')  of  states  in 
F„  we  draw  an  arrow  pointing  toward  q’  which  we  label  with  the  letter 

Cl 

a.  Such  a  graph  is  sometimes  called  the  state  diagram  of  a  nondeter- 
ministic  automaton. 

Two  relational  systems  S  =  (A.Q,  F)  and  S’  =  (A,  Q\  F’)  which 
use  the  same  alphabet  A  are  called  similar  systems.  We  shall  be 
principally  concerned  with  the  interaction  of  similar  systems  and 
this  is  our  reason  for  setting  up  such  an  elaborate  structure.  If  one 
were  concerned  just  with  the  properties  of  a  single  system  S  =  (A,Q,  F) 
he  could  use  the  letters  a  in  A  to  stand  for  the  corresponding  relations 
F  .  Thus,  the  mapping  F  would  not  need  to  be  explicitly  mentioned 
because  it  would  be  placed  on  a  higher  level  of  abstraction  than  is 
now  being  contemplated.  However,  this  cannot  be  our  present  approach. 
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Important  special  cases  are  now  listed. 

1.  We  call  S  determinate  iff  every  relation  F„  is  determinate. 

-  a 

2.  We  call  S  productive  iff  every  relation  F„  is  productive. 

3.  We  call  S  single  valued  iff  every  relation  F„  is  single  valued. 

The  case  in  which  S  is  single  valued  has  been  extensively  investigated 
in  the  literature.  The  following  names  have  been  given  to  this  type 

of  re'  itional  system:  Monadic  aigebra  (J.  B.  Wright),  Unary  algebra 
(J.  R.  Buchi),  Semi  automaton  (A.  Ginsburg).  An  important  fact  which 
was  recognized  by  Wright  and  Buchi  was  that  a  single  valued  relational 
system  is  an  algebra  in  the  sense  of  Garrett  Birkhoff.  Therefore,  a 
host  of  theorems  which  apply  generally  to  algebras  also  hold  for  these 
relational  systems. 

1.  2.  2  Again,  let  us  consider  a  general  relational  system  S  = 

(A,  Q,  F).  We  write  to  represent  the  family  of  all  relations  F  , 

A 

where  a  is  in  A.  Also,  let  V-  stand  for  the  closure  of  under 
Pierce  product.  The  input  semigroup  of  S  is  the  system  whose  car¬ 
rier  is  'r'  +  and  whose  operation  is  Pierce  product.  We  see  that  this 
system  is  indeed  a  semigroup  since  Pierce  product  is  associative  and 
f  is  closed  under  it. 


It  was  apparent  from  section  1.  i.  3  that  I  has  the  properties  of 
a  unit.  Thus  for  all  relations  F^  €  7"  we  have  Iq  °  F^  =  F^  °  Iq  =  F^. 


/  -f*  •  > 

If  we  attach  I  to  we  get  a  set  y*  = 

y 


V;+  o  {L.}  which  is  the 

y 


\ 


♦ 


carrier  cf  the  input  monoid  of  S.  A  monoid  is  defined  as  a  semigroup 
with  a  unit  and  we  see  that  v**  forms  a  monoid  under  Pierce  product. 
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Theorem  5 


To  every  monoid  M  with  a  finite  set  of  generators  G, 
there  corresponds  a  single  valued  relational  system 
=  (G,  M,  F)  whose  input  monoid  is  isomorphic  to  M. 

Proof:  From  the  statement  of  the  theorem,  we  see  that  G  is  to  be  taken 
as  the  alphabet  of  S  and  Mas  its  carrier.  For  each  element  t  e  M, 
we  define  Ft  as  the  set  of  all  pairs  (s,  s’)  such  that  st  =  s’.  The  restric¬ 
tion  of  t  to  G  provides  us  with  the  definition  for  F  of  S^.  Clearly, 
is  single  valued.  We  now  show  that  the  correspondence  t  ~  Ffc 
is  one-to-one.  Let  e  stand  for  the  unit  of  M,  then  (e,  t)  is  in  F^  but 
not  in  Ft,  if  1 4=  t\  Thus,  if  Ft  =  Ft,  we  have  t  =  t\  Also,  Fg  °  Ft 
-  Fgt  since  M  is  associative.  This  shows  that  M  is  isomorphic  to  the 
family  {Fj  under  the  operation  of  Pierce  product.  Finally,  we  prove 
that  '7*  *  is  the  same  as  the  family  {Fj.  However,  Fg  is  in  for 
each  g  €  G  and  G  is  a  set  of  generators  for  M.  Thus,  each  element 
t  €  M  is  either  the  unit  e  or  representable  in  one  or  more  ways  as 


finite  product  gj  •  •  •  Sn  °*  generators.  In  the  former  case,  we  have 
F  =  L-  €  *  and  in  the  latter  cas^  F  =  F  ®  F  0  . . . 

e  ™  *ig2---Bn  gl  g2 

•  F  €  ty*.  Hence,  t1*  is  just  the  family  {fJ,  where  te  M,  and 
gn 

the  theorem  is  proved. 


1.  2.  3  A  finite  sequence  a(l)  a(2)  . . .  a(n)  of  not  necessarily  dis¬ 


tinct  letters  taken  from  an  alphabet  A  is  also  sometimes  called  a  string 
or  word  or  tape  on  the  alphabet  A.  Let  A+  represent  the  set  of  all 
such  strings  on  A.  Two  strings  in  A+  will  be  considered  to  He  equal 


-  »•- *  »*_**?*••  - ■•ygtaragryygyi'inigf. qfr. 
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if  the  letters  occurring  in  corresponding  positions  of  the  strings  are 
always  the  same.  We  define  the  length  of  a  string  a(l)  a(2)  . . .  a(n) 
to  be  the  number  n  of  letter  which  make  it  up. 

The  concatenation  of  two  strings  s  =  a(l)  . . .  a(m)  and 
t  =  h(l)  ...  b(n)  is  the  new  string  st  =  a(l)  ...  a(m)  b(l)  ...  b(n) 
formed  by  juxtaposing  the  two.  Clearly,  concatenation  is  an  associ¬ 
ative  operation.  Also,  A+  is  closed  under  concatenation  and  we  call 
this  system  the  free  semigroup  generated  by  A. 

One  may  attach  a  unit  element  e  to  A+,  giving  a  set  A*  =  A+  u  {e}. 
We  define  e  to  be  an  element  which  follows  the  operation  rules  et  =  te 
=  t  for  all  t  in  A*.  Although  it  is  difficult  to  imagine  e  to  be  a  string 
or  sequence,  we  shall  call  it  the  string  of  length  zero.  It  has  also 
been  variously  called  the  null  sequence,  the  empty  word,  and  possibly 
other  names.  The  set  A*  under  concatenation  and  the  unit  rules  for 
e,  we  call  the  free  monoid  generated  by  A. 

Using  theorem  5,  we  may  construct  ^  =  (A,  A*,  F)  which  is 
called  the  singly  generated,  free  monadic  algebra  on  A.  The  carrier 
A*  of  this  relational  system  is  the  same  as  the  carrier  of  the  free 

monoid  from  which  it  is  derived.  Each  relation  F„  consists  of  all 

a 

pairs  (t,  ta),  such  that  t  is  a  string  in  A*. 

Other  definitions  concerning  A*  will  be  needed  in  our  later  work. 
We  cal'  a  string  r  e  A*  a  prefix  of  a  string  s  t  A*  iff  s  can  be  written 
in  the  form  s  =  rt  for  some  string  t  c  A*.  We  note  that  if  the  string 
t  exists  it  is  unique.  Hence  we  define  r/s  to  be  t  if  s  =  rt  and  take  it 
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to  be  undefined  if  no  such  t  exists.  Similarly,  we  say  t  is  a  suffix 
of  s  iff  s  =  rt  for  some  r.  We  also  introduce  the  notation  s/t  =  r 
as  in  the  previous  definition. 

A  proper  prefix  of  a  string  s  is  a  prefix  of  s  which  is  shorter 
than  s.  Similarly  define  a  proper  suffix.  A  segment  of  s  is  a  prefix 
of  a  suffix  of  s.  It  is  a  proper  segment  if  it  is  shorter  than  s.  We 
note  that  a  segment  of  s  is  also  a  suffix  of  a  prefix  of  s.  In  fact,  s’ 
is  a  segment  of  s  when  we  have  s  =  rs't  for  some  r  and  t.  We  might 
then  use  the  notation  s’  =  r\s/t. 

The  approach  which  we  are  taking  to  the  algebra  of  strings  in 
this  section  may  be  described  as  an  informal  one.  We  simply  set 
down  some  of  the  properties  of  strings  and  rely  on  the  experience 
oi  the  reader  to  justify  their  reasonableness. 

Others  have  taken  more  formal  approaches  which  require  more 
intellectual  effort  and  are  not  necessary  for  our  purposes.  One  such 
approach  is  to  base  the  algebra  of  strings  on  a  simple  set  of  postulates 
which  are  similar  to  the  Peano  postulates  for  numbers.  We  can  then 
derive  such  rules  as  the  associative  law  for  concatenation. 

Another  approach  is  to  treat  each  string  as  a  sequence  in  the 
sense  of  being  a  mapping  from  a  set  of  consecutive  integers  to  A. 

Then  the  mechanism  of  concatenation  is  described  in  terms  of  such 
mappings  and  the  results  are  proved 

1.  2.  4  Let  S  -  (A.  Q,  F)  and  S'  =  (A,  Q’,  F’)  be  two  similar  rela¬ 
tional  systems  which  may  or  may  not  be  distinct.  Then  a  relation 


27 


a  C  Q  x  Q'  is  called  a  generalized  congruence  between  S  and  S’  iff 
for  all  a  e  A, 

1)  I';1otr.Fa'£o> 

ii)  F  '  »Q'>C  F  °  Q  >  ,  and 

a  -  ci 

iii)  «Q>CF  'oQ' 

a  “■  a 

This  formal  definition  can  also  be  expressed  verbally.  Property  (i) 
is  called  the  substitution  property  and  corresponds  to  the  statement 
that  if  (q1#  q2')  is  a  pair  in  a  and  if  (q^  q2)  is  in  Fa  and  (q^\  q2')  is 
in  Fa'  then  (q2>  q2')  is  also  in  a.  Properties  (ii)  and  (iii)  are  dual 
in  the  sense  that  one  may  be  obtained  from  the  other  by  interchanging 
S  and  S"  while  taking  the  inverse  of  a.  Thus,  if  a  is  symmetric,  the 
last  two  properties  are  identical.  We  may  express  (ii)  by  saying 
that  whenever  (q^  q^)  is  in  a  and  there  is  a  pair  (q'p  q'2)  e  Fa\ 
then  there  is  also  a  pair  (q^,  q2)  e  F  .  This  property  is  automatically 
fulfilled  if  S  is  productive.  Similarly  (iii)  holds  automatically  if  S' 
is  productive.  Thus,  if  only  productive  systems  are  being  considered 
we  need  merely  state  (i). 


A  number  of  special  cases  of  a  generalized  congruence  a  are  now 


listed. 


1.  a  is  a  congruence  if  it  is  an  equivalence  relation. 

2.  a  is  a  homomorphism  if  it  is  single  valued.  Special 
types  of  homomorphism  are  now  listed. 

II.  o  is  a  monomorphism  if  a  *  is  determinate. 

4.  a  is  an  eipmorph ism  if  c  *  is  productive. 
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5.  c  is  an  endomorphism  if  it  is  square. 

6.  a  is  an  isomorphism  if  a  *  is  single  valued. 

7.  a  is  an  automorphism  if  it  is  a  square  isomorphism. 
Theorem  6 

If  ffj  C  Q  x  Q’  is  a  generalized  congruence  between  S  and 
S’  and  a,  CQ'x  Q"  is  a  generalized  congruence  between 

£j  - 

S'  andS",  then  a1  o  is  a  generalized  congruence  be¬ 
tween  S  and  S". 

Proof:  To  prove  °  o2  °  Fa"  ^  0  ^2’  we  assume  (q2,  q2' ') 

is  an  element  of  F”1  o  <>  a2  o  Fa”.  Then  find  qr  q1',  and  q^’  such 
that:  (q,q2)e  F&,  (q^q^e  °V  (qi’.qi”)*  and  (qf,  q2")  €  Fa". 
Then,  by  a,  •  F  "  o  Q"  >  C  F  ’  •  Q’  >  we  see  that  since  (q/,  1)  is  in 

ff2  0  Fa"  il  is  111  Fa'  °  Q’  >  •  Hence  there  is  a  q2’  e  Q*  such 

that  (q1',q2')  is  in  Fa'.  Now,  F^  •  01  •  F^Oj,  so  (q2,q2')  is  in 
ar  Similarly,  (q2' ,  q2")  is  in  <r2>  thus  proving  that  (q2,q2M)  is  in 

ax  o  02. 

To  prove  ff,«a2o  Fa”  o  Q"  >  C  Fa  0  Q  >  ,  we  proceed  alge¬ 
braically.  Use  0?  •  Fa '  -  Q”  >  C  Fa'  o  Q'  >  and  obtain  al  •  02  ° 

Fa"  0  Q  >  —  CTl  °  Fa'  °  >  '  since  al  s  Fa'  0  Q'  >  —  Fa  °  ^  >  ’ 

the  result  follows.  The  third  property  for  Uj  °  c2  is  proved  in  a 

similar  way. 

1.  ?.  5  Let  S  =  (A,  Q,  F)  be  a  relational  system.  We  may  extend 

our  notation  by  defining  F  for  each  s  e  A*  as  follows.  When  s  =  e, 

s 

then  let  Fg  =  U.  When  s  is  a  string  a(l)  a(2)  . . .  a(n)  of  length  n  >  0, 
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then  let  Fg  =  t"a(1)a<2). . .  a(n)  =  Fa(l)  0  Fa(2)  0  *  *  *  0  Fa(n)'  Using 
this  rule,  we  see  that  whenever  s  and  t  are  two  strings  in  A*,  then 

F  t  =  Fg  0  Ft>  Consequently,  y?  *  consists  of  just  those  relations 
which  can  be  represented  in  the  form  F  ,  where  s  in  in  A*.  We 
have  thus  extended  F  so  as  to  map  A*  onto 
Theorem  7 

If  o  is  a  generalized  congruence  between  S  =  (A,Q,  F) 
and  S’  =  (A,  Q',  F') ,  then  for  each  s  c  A*, 

i)  F*  o  a  o  F  '  c  a, 

s  s  — 

ii)  aoF'oQ1  >CF  °Q>,  and 

8  —  S 

iii)  c"1  o  Fe  •  Q  >  C  F  '  .  Q'  >  . 
s  s 

Proof:  The  proof  is  by  induction  on  the  length  of  the  string  s. 

Consider  first  the  case  in  which  s  =  e,  so  Fe  =  Iq  and  Fe’  = 

Then  (i),  (ii),  and  (iii)  are: 

l>  >Q  •  « -  V  £  ”• 

ii)  or  o  1^,  o  Q1  >C  ^  s  Q  >  ,  and 

iii)  o' 1  o  lQ  o  Q  >  c  •  Q’  >  . 

These  are  all  trivial. 

Next,  suppose  that  (i),  (ii),  and  (iii)  hold  for  some  string  s  €  A* 

and  let  a  be  any  letter  in  A.  We  then  seek  to  prove  these  same  three 

properties  for  the  string  sa.  Now,  F  =  F  »  F  and  F  ’  =  F  ’  o 


F 


a 


so 


i) 


o  ° 


C  a. 


a  o 


a 


C  F'1 o  o  *  Fq 
—  a  a 
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To  prove 

ii)  we  note  that  F„'*°  Q'  >  Q'  >  ,  so  a  »  F  ’  °  Q’  > 

C  a  .  Fs’  .  Q’  >  C  Fs  •  <k  >  . 

Thus,  il  (q,  1)  is  in  o  °  F  '  .  P  ',Q’  >,  there  must  be  a  pair  (q,  q.) 

e  F  .  Also  there  are  pairs  (q,  q')  e  a  and  (q',  q^’)  €  Fg'.  Hence 

(q1?  Q.^')  is  in  v  by  hypothesis  (i)  where  q^'  is  in  F  '  o  Q’  >  .  Thus 

(q.,  1)  is  in  a  °  F  ’  o  Q’  >  ,  and  hence  in  F  0  Q  >  .  This  shows  that 

(q,  1)  is  in  F  °  F  «  Q  >  .  We  prove  (iii)  similarly.  This  completes 
s  » 

the  inductive  step  and  hence  the  proof. 

Theorem  o 

LfaCQxQisa  generalized  congruence  on  S  =  (A,  Q,  F) 

which  is  symmetric  and  weakly  reflexive,  then  ad£QxW 

W 

is  a  generalized  congruence  between  S  and  a  system  S  = 

W  W 

(A,  W,  F  )  such  that  for  each  a  e  A,  the  relation  F™  is 

_ _  et 

defined  by  F^  -  (a  *  »  F0  °  ad)  n  or  d  o  F  °  ad. 

a  a  ci 

Proof:  We  must  prove  that  ad  has  the  three  properties 

..  -1  d  W  d 

l)  Fa  o  a  *  Fa  C  o  , 

ii)  ad  «  F^  *  W  >  C  F0  o  Q  >  ,  and 

iii)  a'd  o  F0  «  Q  >  C  o  W  >  . 


The  definition  of  F„  may  be  expressed  less  formally  by  saying  that 

a 

W  -d 

(wl’  Wg)  is  Fa  iff  whenever  (w^q)  is  in  a  «  F„,  then  (q,  w2)  as 

in  and  furthermore  such  an  element  q  exis's. 

To  prove  (i),  let  us  assume  that  (q,  w9)  is  in  F  *  u  ad  0  F^. 

-*cl 

An  element  w  ^  must  exist  such  that  (w^,  q)  is  in  a  o  F&  and  (w  w2) 
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T|t  J 

is  in  F.  Hence,  we  see  that  (q,  w0)  is  in  c  and  (i)  is  proved. 

3  Z 

d  W 

To  prove  (ii),  let  us  assume  (q^,  1)  is  in  a  o  Fa  »  W  >  .  There 
are  elements  Wj  and  Wg  such  that  (q^,  Wg)  is  in  o*  and  (Wj,  Wg)  is  in 
F^.  Hence,  there  is  some  element  q  such  that  (w,,  q)  is  in  o ^  <>  F_. 

3  13. 

d 

Therefore,  there  is  an  element  q^’  such  that  (q^Wj)  is  in  a  and 

d  d 

(q/,q)  is  in  F  .  Hence,  (q^q^)  is  in  a  0  cr  =  a  and  since  a  is  a 
generalized  congruence,  it  satisfies  aoF  «Q>CF  «Q>,  Now, 
(q.,  1)  is  in  a  o  F.  «  Q  >  so  it  is  in  F  «  Q  >  ,  thus  proving  (ii), 

13  3 

To  prove  (iii),  let  us  assume  (w,,  1)  is  in  a~ ^  «  F_  o  Q  >  . 

1  3 

Thus,  there  is  at  least  on-  element  q  such  that  (w^,  q)  is  in  a  o  F  . 

Let  us  define  P  as  the  set  oi  nil  such  elements  q.  For  any  two  elements 

q,  q’  in  P  there  are  associated  elements  and  q^  such  that  (q^  w^) 

d 

and  (,q^\  w^)  ai’e  in  aa  and  (q^,  q)  and  (q^',  q’)  are  in  F  .  Thus, 
d  —  d  1 

(q,,  q, ')  is  in  a  o  o  =  o  and  (q,  q')  is  in  F~  °  a  o  F  .  But,  a  is  a 

1*1  3  3 

generalized  congruence  and  therefore  satisfies  F  1  °  a  o  F  C  a. 
Therefore,  (q,  q’)  is  in  a  and  since  q  and  q'  were  arbitrarily  chosen 
from  P  we  see  that  P  is  a  complete  set  with  respect  to  a.  Using 
Zorn's  lemma  as  in  lemma  2  oi  section  1.  1.  6,  we  see  that  P  is  in¬ 
cluded  in  some  maximal  complete  set  K  €  ,  Thus,  there  is  an 

element  Wg  ~  K  such  that  (q,  w^)  is  in  ad  for  each  q  e  P.  But,  P  is 

-d 

just  the  set  of  all  q  such  that  (w-.  q)  is  ia  a  0  F  so  we  therefore 

1  3 

vv  w 

have  the  result  that  (w^w^)  is  in  F&  .  Hence,  (Wj,  1)  is  in  F&  <>  W  > 
and  (iii)  is  proved.  This  completes  the  proof  of  the  theorem. 
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Corollary  1 

If  o  is  a  congruence  relation  on  S ,  then  a d  is  an 

W 

epimorphism  from  S  to  S  . 

Proof:  When  a  is  a  congruence  relation  it  is  an  equivalence  relation 

j 

on  Q.  Hence,  by  remark  4  of  section  1.  1. 6,  we  see  that  or  is  single 
valued.  Since  o  is  not  empty,  the  empty  set  is  not  in  %  and  hence 
eft  is  surjective.  By  theorem  8,  it  is  therefore  an  epimorphism. 


Corollary  2 


X  d 

If  a  is  a  homomorphism  from  S  to  S’,  then  (a  0  o~  )”  »  a 

W 

is  a  monomorphism  from  S  to  S'. 

Proof:  If  a  is  a  homomorphism,  then  o  °  o  *  is  a  congruence 

relation  so  (o  °  a  *)d  is  an  epimorphism.  Write  6  =  (a  <,  a-1) ~d  o  a. 

We  wish  to  show  that  6  o  9~ 1  =  1^  and  1^,.  Now, 

.  - 1  ,  - 1, -d  -1  ,  -l.d  .  -I-d 

6  o  6  =  (a «.  o  )  t-aoa  »(a<>a  )  =(a°cr  )  0 

(a  o  ff“Vd  •  (a  o  or'V  o  (cr  o  a"  Vd  o  (a  o  a'V  =  ^  o 

. ,  - 1  „  -1  ,  -Id  ,  -I-d  -1  -1 

Also  9  o  9-o  o  (ao  a  )  o  (u  o  a  )  0  g  =  o  <>  g  o  g  0 

a  -V*  V"  V 

Corollary  3 

If  g  Ls  an  epimorphism  from  S  to  S',  then 

- 1  -d  W 

(a  $  g  )  »  g  is  an  isomorphism  from  S  to  S'. 

The  proof  of  this  result  is  trivial.  Corollaries  1,  2,  and  3  are 
analogous  to  results  of  a  similar  nature  concerning  congruences  on 
algebras.  If  A  is  an  algebra  and  *  is  a  congruence,  then  it  is  cus¬ 
tomary  to  speak  of  the  quotient  algebra  A  /  s  whose  elements  correspond 
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to  congruence  classes.  In  our  notation,  if  S  is  a  relational  system 

W 

and  a  is  a  generalized  congruence  on  S  then  S  is  the  quotient  sys¬ 
tem.  The  analogy  is  exact  when  S  is  a  monadic  algebra  and  a  is  a 
congruence  on  S. 

1.2.6  Let  S  and  S’ be  two  semigroups.  Then  a  relation  o  C. 

S  x  S '  will  be  called  a  generalized  semigroup  congruence  iff  whenever 
(s^  s^)  is  in  s  and  (s2,  s2')  is  in  <7  then  (s^  s2,  Sj',  s2')  is  in  a.  The 
analogy  between  this  substitution  property  and  that  of  the  previous 
section  should  be  clear.  Properties  (ii)  and  (iii)  in  the  previous 
definition  are  not  applicable  in  this  case  since  the  semigroup  opera¬ 
tion  is  single  valued. 

Remark  5 

The  property  of  being  a  generalized  semigroup  congru¬ 
ence  is  preserved  under  Pierce  product  and  under  in¬ 
finite  set  theoretical  intersection  whenever  these  opera¬ 
tions  are  applicable. 

Proof;  Suppose  and  <j2  are  generalized  semigroup  congruences 
and  ctj  o  a2  can  be  formed.  To  show  that  it  is  also  a  generalized 
semigroup  congruence,  let  (s^,  s^")  and  (s2,s2”)  be  in  *  o^.  Then, 
we  can  find  elements  s^'  and  s2'  such  that  (s  j,  s^')  and  (s2,  s2')  are 
in  while  (Sj',  Sj^")  and  (s2',  s2")  are  in  ct2.  Hence,  (s^Sg.  s’jS'g) 
is  in  and  (s^',  s2',  s^'Sg")  is  in  Thus,  (SjSg,  Sj"s2")  is  in 
<7j  -  o2  and  the  first  half  of  the  remark  is  proved. 
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Next,  suppose  that  Z  is  a  set  of  generalized  semigroup  congru¬ 
ences  a,  all  between  S  and  S7.  Let  (s^,  s^)  and  (s2,  s^)  both  be  in 

a.  Then,  they  are  in  each  a  e  Z.  Consequently,  (SjS2,  s^'Sr,') 
oc  2  ^ 

is  in  each  a  e  Z  so  it  is  in  |  ]  a.  This  completes  the  proof  of  the 

Off  I 

remark. 

Remark  6 

If  a  is  a  reflexive  generalized  semigroup  congruence 

+  * 

on  a  single  semigroup  S,  then  a  =  a  is  also. 

Proof:  This  remark  could  be  proved  easily  if  we  had  closure  under 
set  theoretical  union  as  in  the  case  of  relational  systems.  However, 
such  closure  does  not  hold  in  the  semigroup  case.  We  therefore  as¬ 
sume  (Sp  s^')  and  (s2,  s2’)  are  both  in  o+.  Hence,  for  some  m  and  n, 
we  have  (s^,  Sj^')  e  crm  and  (s2,  s2')  c  <P.  Therefore  (SjSg,  SjS  s2') 
is  in  or  °  cr  -  a  Co  ,  and  the  remark  is  proved. 

Using  the  same  defining  rules  as  we  used  for  generalized  con¬ 
gruences  on  relational  systems,  we  can  define  such  concepts  as  semi¬ 
group  congruences  and  semigroup  homomorphisms,  the  iatter  being 
further  classified  as  epimorphisms,  endomorphisms,  etc. 

Remark  7 

The  mapping  F  of  a  relational  system  S  =  (A,Q,  F).  when 
extended  to  form  a  mapping  from  A*  to  *  is  a  semi¬ 
group  homomorphism  from  the  free  monoid  generated 
by  A  to  the  input  monoid  of  S. 
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Proof:  The  substitution  property  is  just  our  previously  noted  rule: 
Fst  =  Fg  °  Ft  for  all  s,  t  in  A*.  Hence,  this  result  is  obvious. 

1.  2.  7  Let  Q  be  any  set  and  ft  an  operator  mapping  ^(Q)  either 
to  itself  or  another  family  of  sets.  Then,  we  shall  call  ft  a  dis¬ 
tributive  operator  iff  for  every  family  j  of  subsets  T  of  Q  the  equa¬ 
tion 


ft 


(  U  T)  =  U  Si(T) 

rr'  ^  T  ej 


T  e 


holds.  A  special  case  of  the  above  equation  occurs  when  J  is  empty, 
giving  ft  ($)  =  <t>.  The  property  of  finite  distributivity  requires  only 
that  the  defining  equation  hold  for  finite  nonempty  families  J  . 
Remark  8 

A  necessary  and  sufficient  condition  for  ft  to  be 

distributive  is  that  J7(T)  =  ft(q)  for  every 

q  e  T 

subset  T  of  Q. 

Proof:  This  condition  is  clearly  necessary  since  T  is  the  union  of 
singleton  seis  of  its  elements.  It  is  also  sufficient  since 

n(  U  t)  =  a  U  (q;)  =  U  *>(  L  ■  M>  =  U  «(t). 

Te  J  qe  TeJ  T  €  J  q  e  T  TeJ 

Lemma  3 

If  ft  is  a  distributive  operator  mapping  J  (Q)  to  itself 
and  if  ft  *(T)  denotes  the  closure  of  ft  applied  to  the  sub¬ 
set  T  of  Q,  then  ft*(T)  is  the  minimum  of  all  subsets  T' 
of  Q  such  that  T'  T  and  T’  D  ft(T'). 
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Proof:  Abbreviate  ft(ft(. . .  (T). . . ))  as  ft  n(T),  where  n  is  the  num- 

o° 

ber  of  times  &  is  applied.  Then,  we  may  write  ft*(T)  =  I  J  ft  (T), 
ft  n^O 

where  ft  (T)  =  T.  Thus,  we  see  that  ft  *(T)  O  T  and  ft*(T)  £> 

S2(ft*(T)),  so  ft*(T)  satisfies  the  two  given  conditions.  To  show  that 
it  is  a  minimum,  we  take  T'  to  be  any  set  such  that  T*  jD  T  and 
T*  D.  S2(T')*  Let  q  be  an  element  of  ft  *(T).  Then  q  is  in  ft  n(T)  for 
some  n  >  0.  Hence,  for  some  sequence  qQ,  q^, . . . ,  qn  =  q,  we  have 
q^  e  T  and  qi  €  ft (q.^)  for  i=l, . . . ,  n  by  remark  8.  Thus,  by  induc¬ 
tion,  we  see  that  q^  is  in  T'  and  each  q^  is  in  T'  since  q^  ^  is  in  T'. 
Hence,  each  term  in  the  sequence  is  in  T',  and  q  is  therefore  in  T'. 
This  completes  the  proof  since  we  have  shown  that  ft*(T)  is  a  subset 
of  any  T\ 

Remark  0 

Given  a  pair  S,  S'  of  similar  relational  systems,  not  neces¬ 
sarily  distinct,  and  a  relation  oQ  C_  Q  x  Q',  then  there  is  a 
unique  minimum  relation  u  C  QxQ'  satisfying  (i)  a  D  oQ, 

and(ii)  a  D  F'1  «  a  *  F 

a  e  A 

If  we  define  ft(X)  =  1  J  f’1  ,  X  .  F  '  then  a  =  ft*(on).  Also,  if 

arA  a  a  0 

there  is  any  generalized  congruence  o'  between  S  and  S'  such  that 

o'  D  Oq  ,  then  ois  the  minimum  such  generalized  congruence. 

Proof :  The  first  part  of  this  remark  follows  from  lemma  3  and 

the  observation  that  the  operator  ft  of  the  remark  is  distributive. 

The  second  part  of  the  remark  can  be  proved  by  first  noting  that 
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a  c_  a*  since  a  is  minimal  over  relations  satisfying  (i)  and  (ii).  There¬ 
fore,  (J#F  'oQ'>Cff'«F  '«Q'>C  F  *  Q  >  .  Similarly,  we 

a  "  a  A 

show  that  a o  F  o  Q  >  c  F  '  o  Q'  >  .  Thus,  a  is  a  generalized 

a  a 

congruence  and  the  proof  is  complete. 

We  may  think  of  remark  9  as  providing  us  with  an  algorithm 
for  constructing  <j  =  0*(<Tq)  when  Oq  is  given.  To  carry  out  this  algo¬ 
rithm,  we  define  ai+^  =  u  n(o^)  =  u  1^1  Fa*  0  a.  «*  Fa',  for 

a  e  A 

i=0,  1,  2, . . .  .  Then  each  element  of  a  is  contained  in  some  While 
this  algorithm  appears  to  be  infinite,  there  are  many  important  cases 
in  which  it  may  be  terminated  after  some  finite  number  n  of  steps. 

In  particular,  if  or  ^  =  aQ,  then  clearly  =  a .  It  is  also  clear  that 
when  each  F  and  each  F  '  is  finitary,  the  algorithm  terminates.  A 
fairly  obvious  special  case  of  this  situation  occurs  when  the  relational 
systems  S  and  S'  are  finite,  i.  e.  Q  and  Q'  are  finite. 

Theorem  9 

Let  S  =  (A,  Q,  F)  and  S'  =  (A,  Q',  F')  be  similar  relational 
systems  and  let  ^CQxQ'bea  relation  such  that  for 
all  a  e  A,  an  o  F  '  »  Q'  >  C  F_  °  Q  >  and 
a~ 1  o  F0  °  Q  >  C  F  '  «  Q1  >  .  Define  R(X)  = 

U  gL  a 

F  •Xo(F')'1.  Thena=  n*(o0’)  is  the  unique 

a  e  A 

maximum  generalized  congruence  relation  between  S  and 
S'  such  that  a  C  Oq.  If  S  =  S'  and  is  symmetric,  then  a 
is  symmetric.  If  S  =  S'  is  single  valued  and  Oq  is  an 
equivalence  relation,  then  o  is  a  congruence  relation. 
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Proof:  The  operator  ft  as  defined  in  the  theorem  is  clearly  a  dis¬ 
tributive  operator.  By  lemma  3,  we  see  that  a  is  the  minimum  rela¬ 
tion  satisfying  a  O  an  and  F  iffo(F  ')  for  each  a  e  A.  The 

condition  a  o_  a0  may  be  rewritten  oC_  o^.  Hence,  a  °  Fa'  o  Q'  > 

C_  o0Fa'  o  Q'  >  C.  Fa  •  Q  >  .  Also,  a  1 C  aj 1 ,  so  a" 1  o  Fa  o  Q  > 

C  °q  1  o  Fa  o  Q  >  C_  Fa'  o  Q’  >.  The  condition  FR  •  a  »  (Fa')  1  C 
a  is  equivalent  to  F"1  o  a  °  F  'Ca,  by  the  following  argument : 

F  o  a  o  (F  ’)_1  C  a  <=>  Tr(F  •  a  »  (F  ')-1  o  a"1)  =  0  <==> 

Tr(a  o  F  '  °  a"1  •  F"1)  =  0  <=>  Tr(FQ_1  o  a »  F  ’  .  d'1)  =  0  <=> 

a  3  a.  a. 

F_1<*aoF  'Ca.  Thus,  the  two  conditions  are  equivalent  to  a  being 
a  a 

a  generalized  congruence  such  that  a<L  Oq.  Since  a  is  a  minimum, 
we  see  that  a  is  the  maximum  such  generalized  congruence. 

Next,  we  assume  that  S  -  S’  and  that  aQ  is  symmetric.  Hence, 

Oq  is  also  symmetric.  Define  recursively,  oT^~=  <T  u  ftlm),  for 
i=0, 1, . . .  .  Taking  a  as  symmetric,  so  is  ft  (a.)  from  its  defini¬ 
tion,  so  is  symmetric,  completing  the  inductive  step.  But,  if 
(q,  q')  is  in  a  then  (q,  q')  is  in  a.  for  some  i  and  hence  (q\  q)  is  in 
o7  C  a.  Thus,  a  is  symmetric  so  a  is  symmetric. 

Finally,  we  assume  that  S  =  S'  is  single  valued  and  that  aQ  is  an 
equi\alence  relation.  Thus,  is  irreflexive.  Let  us  assume  in¬ 
ductively  that  oT  is  irreflexive.  Then  F  °  a.  °  F„  *  is  irreflexive 
l  a  i  a 

because  F  is  determinate.  Hence,  ft(oT)  ’s  irreflexive  and  so  is 

a  * 

&~~T  .  Since  each  element  of  a  is  in  some  c,  ,  we  see  that  a  is  irre- 
i+1  i 

flexive  and  thus  a  is  reflexive.  Now  a^  is  transitive  so  a^  °  aQ  C  0q- 
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Because  of  the  symmetry  of  Oq,  this  property  is  equivalent  to 
°0  °  °0  -  °0  tlie  foll™inS  argument  1  Oq  0  CTq  C_  Oq  <— > 

Tr(°Q  0  °o  °  =  0  <==>  Tr^n  0  an  1  0  °i~  *)  =  0  <==> 


'0  0  °  u0 


Tr(°o 1 0  ao  °  ao^  =  0  <==>  ao 1 0  ao  ao  <==>  ao 0  ao  —  ao  * 


"0  0  0 


Let  us  assume  inductively  that  o.  is  transitive.  We  wish  to  prove 

that  a.  ,  is  transitive  so  we  write  ct.  ,  «  ct.  ,  =  ct.  ,  o 
i--l  l+l  l+l  l+l 


(57uU 


UFa  "  ffi  ‘  ^  *  ai+l  *  Ji  u  U/1+l '  Fa  •  °i  *  Fal- 
a  e  A  aeA 


Now,  j.  .j,  c_  cr,  so  a.+1  o  a"  c_  a  o  aT  oT  by  the  transitivity  of  ct.. 

Also,  Fa  0  oT  °  Fft1  C_  a,  +  j  by  the  definition  of  cr.  ^  and  hence 

Tr(F,  °  ct.  o  F„  1  o  a.  ,)  =  0  giving  Tr(F_  1  «  cr.  ,«F  »a)  =  0 

v  a  i  a  i+l'  b  b  'a  l+l  a  i' 

<= >  F " 1  o  cr.  .  o  F_  C  ct..  But  F0  is  productive,  so  I  C  F„  ®  F~*. 

3,  1+1  a  1  a.  -  ci  a 

Hence  a  +  1  •F1CP>.  F^1  .  .  Fa  C.  Fa  .  Thus, 

U/i+l  •Fa>VF;‘c  U  Fa  •  °i  °  °i  °  Fa^—  UFa  0  °i  *  Fa 

aeA  aeA  aeA 


This  gives  us  o\+1  0  °i+i  Sl  ai  u  v_J 


aeA 


F„  °  o.  o  F_1  =  o.  .  .  Hence, 
a  i  a  i+l  ’ 


ct.  is  transitive  completing  our  inductive  step  and  the  proof  of  the 
theorem. 


As  was  the  case  with  our  previous  result,  remark  9,  this  theorem 
m;iy  also  be  regarded  as  defining  an  algorithm  for  finding  a  congru¬ 
ence  r  elation  a  with  the  properties  described  in  the  theorem.  It  is  true 
that  the  algorithm  may  not  always  be  finite,  but  in  many  practical  cases 
it  may  be  shown  to  terminate  after  a  finite  number  of  steps.  We  shall 
call  this  the  Moore  algorithm  since  the  process  was  invented  by  E.  F. 
Moore  for  the  case  in  which  S  is  a  finite  monadic  algebra  and  0q  is  an 


.lafcaiiiMwiiafa 
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equivalence  relation.  In  this  case  a  is  a  congruence  relation  and  the 

process  terminates  after  a  finite  number  of  steps.  Moore  was  able 

d  W 

to  construct  a  which  is  an  epimorphism  and  S  which  is  the  image 

d  W 

system  under  a  .  He  called  S  the  "reduced  machine. " 

1.  2.  8  A  partial  ordering  on  a  set  Q  was  defined  in  section  1.  1. 3 
as  a  relation  a  c_  Q  x  Q  which  is  reflexive,  antisymmetric,  and  tran¬ 
sitive.  One  example  of  a  partial  ordering  is  the  relation  C_  of  set  in¬ 
clusion  among  a  family  J  of  subsets  of  a  set  T.  We  see  that  the 
relation  TV  C_  T\  between  two  such  subsets  must  be  reflexive,  anti¬ 
symmetric,  and  transitive.  Therefore,  it  is  a  partial  order.  This 
means  that  any  family  of  relations  on  the  same  pair  Q  x  Q'  of  sets  is 
partially  ordered  by  set  inclusion.  However,  other  examples  of  par¬ 
tially  ordered  sets  abound  in  mathematics  and  in  everyday  experience. 

KffC_QxQisa  partial  ordering  and  P  is  a  subset  of  Q,  then 
we  say  that  q  is  an  upper  bound  of  P  iff  (p,  q)  is  in  o  whenever  p  is  in 
P.  Similarly  ,  we  say  q'  is  a  lower  bound  of  P  iff  (q',  p)  is  in  a  when¬ 
ever  p  is  in  P.  A  set  P  may  or  may  not  possess  upper  bounds  and 
lower  bounds  when  a  is  any  given  partial  ordering.  However,  an  im¬ 
portant  special  case  occurs  when  every  nonempty  set  P  not  only  has 
an  upper  bound  and  a  lower  bcuna  but  it  has  a  unique  minimum  upper 
bound  and  a  unique  maximum  lower  bound.  A  partia.  ordering  of  this 
type  *s  called  a  complete  lattice.  Specifically,  there  is  a  unique  upper 
bound,  railed  the  supremum  of  1/  and  written  Sup(P),  which  has  the 
property  that  if  q  is  any  upper  bound  of  P,  then  (Sup  (P),  q)  is  in  <?. 
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Also,  there  is  a  unique  lower  uound,  called  the  infimum  oi  P  and  written 
Inf  (P),  which  has  the  property  that  if  q'  is  any  lower  bound  of  P,  then 
(q',  Inf(P))  is  in  a. 

In  case  it  is  only  possible  to  assert  that  Sup  (P;  and  Inf  (P)  exist 
when  P  is  finite,  we  say  simply  that  a  is  a  lattice ,  the  adjective  "com¬ 
plete"  being  reserved  for  lattices  in  which  it  is  known  to  be  true  in 
general. 

A  notation  that  is  frequently  used  with  lattices  is  to  write  p  u  q 
and  p  n  q  to  represent  the  supremum  and  infimum  respectively  of  the 
two  element  set  {p,  q}.  A  great  disadvantage  of  this  notation  is  that 
the  same  symbols  are  used  for  set  theoretical  union  and  intersection. 
When  we  discuss  the  partial  ordering  among  sets  in  a  family  this  no¬ 
tation  is  clearly  ambiguous  because  the  set  theoretical  and  lattice  the¬ 
oretical  operations  may  not  signify  the  same  result.  It  will  be  neces¬ 
sary  to  avoid  such  ambiguity  by  carefully  explaining  which  type  of  opera¬ 
tion  is  intended  in  cases  of  uncertainty. 

A  great  convenience  of  the  p  u  q  and  pn  q  notation  is  that  one 
may  derive  simple  algebraic  rules  which  apply  to  lattices.  The  reader 
may  verify  the  following: 

1.  p  n  (q  n  r)  =  (p  n  q)  n  r  and  p  u  (q  u  r)  =  (p  u  4)  u  r,  associativity. 

2.  p  n  q  =  q  1  p  and  p  u  q  =  q  u  p,  commutativity. 

3.  p  1  p  =  p  and  p  u  p  =  p,  idempotenee. 

4.  p  n  (p  u  q)  =  p  and  p  u  (p  n  q)  =  p,  absorption. 
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Many  other  identities  may  be  derived.  It  is  also  possible  to  regard 
the  operations  p  u  q  and  p  n  tj  as  fundamental  and  from  their  prop¬ 
erties  derive  the  properties  of  partial  ordering  in  a  lattice.  For 
this  purpose  we  may  define  a  by  either  of  the  two  rules : 

5.  p  n  q  =  p  <=  >  (p,  0}  e  a,  or 

5'.  p  u  q  =  q  <=>  (p,  q)  e  a  ,  both  of  which  are  equivalent. 

By  introducing  this  algebraic  notation  we  have  changed  our  em¬ 
phasis  from  the  set  a  to  the  set  Q.  From  the  point  of  view  of  the 
operations  u  and  n  ,  a  lattice  is  an  algebra  whose  carrier  set  is  Q. 

When  the  entire  carrier  set  Q  of  a  lattice  has  a  supremum  and 
an  infimum  these  elements  are  written  I  and  O  respectively.  We 
note  that  Sup  (Q;  =  I  and  Inf  (Q)  =  0  always  exist  in  the  case  of  com¬ 
plete  lattices.  These  elements  obey  the  special  rules : 

6.  O  il  p  =  O  and  i  up  =  I. 

7.  O  U  p  =  p  and  In  p  =  p. 

Lf  J  is  a  family  of  subsets  of  a  set  T  and  J  forms  a  lattice 

er  set  inclusicn,  then  the  relation  T;  C  T.  between  members  of 

*  J 

KJ  has  the  s"  me  meaning  in  the  lattice  and  in  the  family  of  sets 
j  .  However,  the  set  theoretical  union  and  intersection  of  a  sub¬ 
family  /j  '  of  J  ,  which  we  shail  write  as  I  )  T'  ana  f  ;  T 

Ti  ,  A  ♦  .  1  » 

e  1  t 

respectively  may  not  be  the  same  as  the  lattice  theoretical  Sup  („ '  ’ 
an  t  Inf  {J  '). 
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Remark  10 


We  always  have  1  J  T '  c.  Sup  (J  ')  and  Inf  (J  ’)  < 

T-rj’  ~~ 

T\  if  Sup  (J  ')  and  Inf  (J  ')  exist.  However, 


T’  e  J  ’ 

when  ii  T'  is  in  J  then  it  is  Sup  (  J  '),  and  when 
T '  €  J  ' 

f^|  T’  is  in  J  then  it  is  Inf  '). 

T'  e  J  ’ 

Proof:  Let  T’  be  any  element  of  J  Then  T’  £_  Sup  (J  ')  by  the 
definition  of  Sup  ■  ’).  Hence,  taking  the  union  over  ',  we  obtain 
(I  T'  c.  Sup  ').  If  T’  is  in  J  we  see  that  it  is  an 

Tt  /  T  rp  r  y  T 

€  v:  I  t  c-, 

upper  bound  of  \  Hence,  T'  Sup  {-J  ')  by  the  defini- 


T'  e 


tion  of  Sup  (>,'  ’).  This  yields  [)  T'  =  Sup  (J  ')  and  one  half  of 

T'Vj' 


the  remark  is  proved.  The  second  half  may  be  proved  by  a  dual  argu¬ 
ment. 


Remark  11 

The  family  of  all  generalized  congruences  a  between  two 
relational  systems  S  and  S’  forms  a  complete  lattice  under 
set  inclusion. 


Proof:  Since  this  set  of  generalized  congruences  is  closed  under  set 
theoretical  union  and  intersection,  we  may  use  remark  10  to  show  that 
the  supremum  and  infimum  of  any  family  of  generalized  congruences 
between  S  and  S'  must  be  just  the  union  and  intersection  respectively. 
Thus,  the  proof  follow;.. 
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The  following  theorem  is  a  generalization  of  a  result  due  to 
E.  H.  Moore  (not  E.  F.  Moore)  and  is  useful  in  determining  which 
partially  ordered  sets  are  lattices. 

Theorem  10 

IfaC^QxQisa  partial  ordering  of  Q  such  that  every 
nonempty  subset  P  of  Q  has  an  infimum  and  Q  has  an 
upper  bound  I,  then  a  is  a  complete  lattice. 

Proof :  To  prove  this  theorem,  we  must  show  that  every  nonempty 
subset  P  of  Q  has  a  supremum.  Let  P'  be  the  set  of  upper  bounds  of 
P.  We  see  that  P’  is  nonempty  since  I  is  an  upper  bound  of  P  and 
therefore  a  member  of  P\  Hence,  Inf  (P')  exists  and  by  its  defini¬ 
tion,  we  see  that  (q,  Inf  (P'))  is  in  a  whenever  q  is  in  F  so  Inf  (P') 
is  in  P\  Also,  (Inf  (P')  ,  q’)  is  in  a  whenever  q*  is  in  P’,  so  Inf  (P') 
=  Sup  (P).  This  completes  the  proof  of  the  theorem  since  P  was  cho¬ 
sen  arbitrarily. 

It  may  be  noted  that  the  dual  of  theorem  10  may  be  stated  ?.nJ 
is  equally  valid. 

Given  two  partial  orderings  .  Q^x  Qx  and  C  Q2  *  Q2 

such  that  Qj  C_  Q2  and  er^  C_  Qg,  we  say  that  Oj  is  a  subordering  of 

r/g.  However,  if  cr^  and  0^  are  both  lattices  we  would  not  generally 

assert  that  is  a  sublattice  of  because  we  regard  a  lattice  as  an 

algebra.  To  illustrate  this  difference,  let  p  0  and  p  q  repre- 

1  '  2 

sent  the  supremum  of  {p,  qf  in  and  respectively.  Clearly. 

(p  q,  p  q)  is  in  a9,  but  we  cannot  generally  conclude  that  p  q 
2  1  £  1 
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and  p  u  q  are  equal.  The  same  comment  applies  to  the  infimum. 

2 

Thus,  we  shall  only  say  that  is  a  sublattice  of  Og  when  the  supre- 
mum  and  infimum  of  any  set  in  the  first  system  are  the  same  as  the 
supremum  and  infimum  of  that  set  in  the  second. 

The  following  two  theorems  are  analogous  to  results  obtained 
by  Garrett  Birkhoff  concerning  congruence  relations  on  algebras 
Theorem  11 

The  family  0  of  equivalence  relations  9  C_  Q  x  Q  on  a 

set  Q  forms  a  complete  lattice  under  set  inclusion.  If 

C,  is  a  nonempty  subset  of  (0)  ,  then  Inf  ( 9 

0  6  C 

and  Sup  (  d,  )=({_]  9 )*. 

9e  (!■ 

Proof :  We  note  that  if  d  is  a  nonempty  set  of  equivalence  rela¬ 

tions,  then  0  9  is  an  equivalence  relation.  Therefore. 

9e  cl 

f  )  9  =  Inf  (  C..  )■  Furthermore,  Qx  Q  is  an  equivalence  relation 

so  (0)  has  an  upper  bound.  Hence,  by  theorem  10,  (0)  is  a  com¬ 
plete  lattice  under  set  inclusion.  It  is  also  not  difficult  to  see  that 

Sup  (  £  )  =  (  9)*  because  Sup  (  £  )  9.  Since  0  J  9)* 

6t  ('.  Bed  9t  (1 

is  reflexive  and  symmetric  and  is  the  minimum  transitive  relation 

including  9 ,  it  is  the  minimum  equivalence  relation  including 

0  e  r. 

0  ,J  9.  Thus  the  theorem  is  proved. 

9t  C 
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Theorem  12 

The  family  Z  of  congruence  relations  a  C  Q  x  Q  on  a 
relational  system  S  =  (A,  Q,  F)  forms  a  complete  lattice 
under  set  inclusion  which  is  a  sublattice  of  the  lattice  of 
equivalence  relations  on  Q. 

Proof :  Assume  that  Z  is  nonempty  so  that  the  lattice  of  congruence 

relations  if  it  exists  is  nondegenerate.  Then,  if  c  is  a  nonempty 

family  of  congruence  relations  cron  S,  we  see  that  a  is  a  con- 

ae  (I 

gruence  relation  because  it  is  both  an  equivalence  relation  and  a  gen¬ 
eralized  congruence.  Also,  (l^J  o)*  =  (  o)+  since  each  <je 

(X€  <?■  a  e  o' 

is  reflexive.  From  our  results  concerning  closure  of  generalized 
congruences  under  union  and  transitive  extension,  we  see  that 

o)  is  a  generalized  congruence  and  since  it  is  an  equivalence, 

CT€  c1 

it  must  be  a  congruence.  Hence,  both  Sup  (£)  and  Inf  ( J )  are  in  Z  . 
so  it  forms  a  sublattice  of  the  lattice  of  equivalence  relations,  and 
the  theorem  is  proved. 

It  is  interesting  to  compare  theorem  .12  with  remark  11.  From 
remark  11  we  see  that  the  family  of  generalized  congruences  on  a  sys 
tern  S  forms  a  complete  lattice  under  set  inclusion.  The  subset  of  this 
lattice  consisting  of  all  congruences  is  also  a  complete  lattice  under 
set  inclusion.  However,  the  second  lattice  is  not  generally  a  sublattice 
of  the  first,  it  is  clear  that  the  reason  is  that  the  union  of  a  set  of  equiv¬ 
alence  relations  is  not  generally  transitive  and  hence  not  another  equiva¬ 
lence  relation.  Therefore  the  Sup  (  CJ)  over  the  lattice  of  generalized 
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congruences  may  differ  from  the  Sup  ( £,)  over  the  lattice  of  con¬ 
gruences. 


Section  2 


Use  of  Multiple  Index  Matrices  in  Generalized  Automata  Theory 
2.  1  Introduction 

An  important  generalization  of  automata  theory  occurs  when  one 
allows  the  transition  function  of  an  automaton  to  have  several  argu¬ 
ments.  An  ordinary  automaton  has  a  function  f  associated  with  each 
input  configuration  o  which  maps  the  present  state  i  to  the  next  state  j. 
However,  in  the  generalization,  these  functions  may  have  several 
arguments  i(l), . . . ,  i(p)  rather  than  a  single  argument  i.  Mezei 
and  Wright  [31],  and  Thatcher  and  Wright  [42]  have  shown  that  many 
of  the  results  of  automata  theory  carry  over  in  this  generalization. 

It  is  no  longer  reasonable  to  regard  the  arguments  i(l), . . . ,  i(p) 
as  states  of  a  machine,  since  we  may  think  of  a  machine  as  possessing 
only  a  single  state  at  a  given  time.  Nevertheless,  it  is  likely  that  the 
new  automata  theory  will  have  greater  and  more  direct  applicability  to 
machines  than  the  old,  because  the  operations  actually  carried  out  by 
computers  do  usually  have  several  arguments,  and  it  is  much  more 
natural  and  convenient  to  describe  a  computer  in  terms  of  the  opera¬ 
tions  it  can  perform  than  by  means  of  a  state  transition  table  or  diagram. 

Difficulties  with  this  generalization  have  not  been  so  much  concep¬ 
tual  as  notational.  Thus,  with  the  object  of  placing  the  entire  theory 
within  the  framework  of  modern  mathematics,  Eilenberg  and  Wright 
[7]  have  represented  the  generalized  automata  as  categories,  drawing 
heavily  upon  earlier  work  of  Lawvere  [  26 ]  in  universal  algebra. 
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In  this  paper  we  wish  to  describe  an  alternative  method  of 
systematizing  the  treatment  of  generalized  automata  in  a  way  which 
we  believe  presents  a  number  of  advantages  *. 

I  (1)  The  method  is  based  on  familiar  mathematical  notions  which 

I  are  used  frequently  in  switching  and  automata  theory.  It  does 

not  require  the  introduction  of  category  theory  although  category 
theory  may  be  applied  to  this  as  to  many  other  mathematical  sys¬ 
tems. 

* 

■  (2)  There  is  a  convenient  way  of  visualizing  the  constructions  of 

» 

the  system  which  enables  one  to  anticipate  results. 

(3)  The  present  approach  is  in  some  respects  similar  to  the  product 
and  permutation  category  (PROP)  method  described  by  MacLane 
[29],  but  is  based  upon  a  single  operation  and  permits  the  deri¬ 
vation  of  such  notions  as  direct  product  and  its  properties  in 
terms  of  this  operation.  Also,  there  is  complete  left-right  duality 
in  the  theory. 

(4)  There  is  a  natural  generalization  to  relational  systems  which 
does  not  require  the  assumption  of  a  distributive  law  with  re¬ 
spect  to  set  union.  A  further  possibility  of  generalization  has  to 
do  with  matrices  of  numbers. 
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2.  2  Multiple  Index  Matrices 

If  A  is  a  matrix,  then  ar  is  often  written  to  represent  the  ele- 
ment  in  the  i-th  row  and  the  j-th  column  of  A.  However,  to  avoid 
cumbersome  subscripts,  we  shall  write  [i(l)  ...  i(p)  aj(l)  . . .  j(q)] 
to  represent  the  matrix  element  when  there  are  p  row  indices 
i(l), . . . ,  i(p)  and  q  column  indices  j(l), . . . ,  j(q).  Our  concern  will 
be  with  matrices  in  which  all  indices  have  the  same  range  K,  where 
K  is  assumed  to  contain  at  least  two  elements.  The  matrix  elements 
are  the  Boolean  quantities  1  or  0,  i.  e.  truth  or  falsity,  but  most  of 
what  we  have  to  say  applies  equally  well  to  matrices  whose  elements 
are  numbers. 

We  shall  call  the  above  matrix  A  a  (p,  ql  index  matrix  and  allow 
p  and  q  to  be  any  integers  greater  than  or  equal  to  zero.  When  p  -  0 
and  q  >  0  we  call  A  a  row  vector  and  when  p  >  0  and  q  =  0  we  call  it 
a  column  vector.  If  p  =  q  =  0,  it  is  called  a  scalar. 

An  alternative  terminology  derived  from  network  theory  is  to 
call  p  and  q  the  numbers  of  input  and  output  lines  respectively.  Wa 
can  thus  represent  A  as  a  box  in  the  diagram  below. 
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We  interpret  this  diagram  as  a  device  or  process  A  which  re¬ 
quires  p  signals  or  arguments,  taken  from  K  and  denoted  by  i(l), . . . , 
i(p),  on  the  input  lines  1, . . . ,  p  respectively.  It  produces  some  con¬ 
figuration  of  q  signals  or  results,  also  taken  from  K  and  denoted  by 
j(l), . . . ,  j(q),  on  the  corresponding  output  lines,  such  that  the  matrix 
element  [i(l)  ...  i(p)  aj(l)  . . .  j(q)]  has  the  value  1,  i.  e.  is  a  true 
proposition.  The  sequence  j(l), . . . ,  j(q)  may  not  be  precisely  speci¬ 
fied  by  this  requirement  just  as  the  "next  state"  of  a  nondeterm inistic 
machine  is  not  precisely  specified,  and  may  be  regarded  as  depend¬ 
ing  upon  unspecified  data  or  conditions. 

Our  basic  rule  concerns  the  formation  of  matrix  product.  Let 
A  and  A'  be  (p,  q)  index  and  (p’,  q')  index  matrices  respectively.  Then 
the  matrix  product  AA'  =  B  is  defined  by  treating  two  cases. 

Case  1.  If  q  >  p',  then  B  has  p  row  indices  and  q”  =  q'  +  p  -  p’  column 
indices.  Each  element  of  B  is  defined  by  the  rule:  [i(l)  . . .  i(p)  bj(l) 

•  •  •  j(q”)l  =  •  •  •  UP)  ak(l)  . . ,  k(p')j(q’+l)  . . .  j(q")l  [k(l)  •  •  • 

k(p')  a'j(l)  . . .  j(q')i  k(l)  ...  k(p')  e  K.  A  diagram  representing  this 
case  is  shown  below. 
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Case  2.  If  q  <  p’,  then  B  has  p"  =  p  +  p’  -  q  row  indices  and  q'  column 
indices.  Each  element  of  B  is  defined  by  the  rule*.  [ i(  1)  ... 

Up")  bi(  1)  ...  j(q')]  =  2[i(l)  ...  i(p)  ak(  1)  ...  k(q)j  [k(l)  ... 
k(q)  Up+1)  . . .  i(p")  a'j(l)  ...  j(q'))  k(  1)  . . .  k<q)  e  K. 

In  this  case  we  have  the  following  diagram. 


When  working  with  Boolean  matrices,  we  interpret  sum  and 
product  as  union  and  intersection  in  the  above  expression.  When  using- 
strict  logical  notation  if  one  regards  each  matrix  element  as  a  prop¬ 
osition,  one  should  replace  the  symbol  2  by  3,  the  symbol  =  by  <=>  , 
and  insert  &  between  the  two  elements  on  the  right. 

2.  3  Linked  Lines  and  Reduced  Matrices 

Let  A  be  a  (p,  q)  index  matrix.  Then,  we  shall  say  that  the  m-th 
row  index  is  linked  with  the  n-th  column  index  if  and  only  if 

( 1)  [  i(  1)  . . .  i(p)  aj(  1)  . . .  j(q)J  =  0  whenever  i(m)  /  j(n),  and 

(2)  [  i(  1)  ...  i(p)  aj(l)  . . .  j(q)]  is  independent  of  x  -  i(m)  =  j(n) 
whenever  i(m)  =  j(n). 
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If  the  p-th  row  index  and  the  q-th  column  index  are  linked,  then 
we  may  reduce  A  to  a(p-l,  q-1)  index  matrix  A  ^  in  which  the  element 
[  i(  1)  ...  i(p-l)  a^l)  ...  j(q-l)]  is  simply  taken  as  equal  to  the  element 
[i(l)  ...  i(p-l)x  aj(l)  . . .  j(q-l)x]  which  is  the  same  for  all  x  m  K.  A 
reduced  matrix  is  one  in  which  the  p-th  row  index  and  q-th  column  in¬ 
dex  are  not  linked.  It  is  clear  that  to  any  matrix  A  there  corresponds 
a  unique  reduced  matrix  Af  obtained  by  repeatedly  eliminating  the 
highest  numbered  row  and  column  indices  until  this  is  no  longer  possible. 

Two  matrices  may  be  regarded  as  equivalent  if  they  have  the 
same  reduced  matrix.  It  is  possible  to  show  that  this  equivalence  rela¬ 
tion  is  actually  a  congruence  relation  with  respect  to  matrix  multipli¬ 
cation.  That  is  to  say,  we  obtain  equivalent  results  regardless  of 
whether  we  multiply  two  matrices  or  their  reductions.  Thus,  in  the 
remainder  of  this  abstract  we  shall  work  with  these  congruence  classes 
of  matrices,  generally  using  reduced  matrices  as  representatives  of 
Liieir  classes.  In  particular,  we  shall  take  the  numbers  p,  q  associated 
with  any  class  to  be  those  of  the  reduced  representative  of  that  cl-  ss 

A  diagram  corresponding  to  a  congruence  class  of  matrices  may 
be  drawn  as  follows. 
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The  box  labelled  A  corresponds  to  the  reduced  representative  of  the 
class.  Above  this  box  we  imagine  an  infinite  number  of  lines  corres¬ 
ponding  to  the  linked  indices  of  other  members  of  the  class.  Any  in¬ 
put  signal  or  argument  on  a  lwe  numbered  n  >  p  will  pass  unchanged  as 
a  result  on  the  linked  output  line  numbered  n-  p+q. 

2.  4  The  Group 

Let  U  be  the  group  of  all  finite  permutations  of  the  set  of  all 
natural  numbers  {1,  2, . . . }.  We  can  construct  a  set  ft  of  multiple 

t 

index  matrices  which  is  isomorphis  to  ft  with  matrix  multiplication 
corresponding  to  the  group  operation.  This  construction  is  possible 
for  any  given  index  range  K  of  cardinality  two  or  greater. 

Let  S  be  a  permutation  in  wh.ch  leaves  all  numbers  greater 
than  n  fixed  and  maps  { 1,  2, . . . ,  n}  to  {s^,  . . . ,  s^} .  We  construct 

a  corresponding  matrix  H  with  p  =  q  =  n  whose  matrix  elements 
[i(l). . .  i(n)  hi(Sj). . .  i(sn)]  are  1  for  any  sequence  i(l), . . . ,  i(n)  in  K, 
and  whose  other  matrix  elements  are  all  0. 

It  may  be  shown  that  each  row  index  number  m  of  His  linked  to 
the  column  index  numbered  sffi,  Thus,  on  our  diagram  H  is  an  opera¬ 
tion  corresponding  to  a  scrambling  of  the  lines  when  passing  from  in¬ 
put  to  output  as  given  by  the  permutation  S.  It  is  possible  to  prove 
that  matrix  multiplication  of  members  of  9  corresponds  to  the  product 
of  the  permutations.  Note  that  the  members  of  *v  are  not  permutation 
matrices  in  the  usual  sense  since  they  do  not  permute  the  members  of 


K  but  rather  the  indices. 
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As  a  convenient  set  of  generators  for  ft  we  shall  choose  the 

matrices  7rn  corresponding  fo  the  cyclic  permutations  (12  . . .  n)  of 

the  first  n  natural  numbers.  We  shall  write  7r^  for  the  m-th  power 

of  77  .  A  diagram  for  7rm  may  be  drawn  as  follows, 
n  n 


2.  5  Relational  Systems 

The  importance  of  the  group  in  connection  with  multiplication 
of  multiple  index  matrices  should  be  clear  since  by  choosing  a  suitable 
member  H  of  )/  we  may  make  any  connection  AHB  of  output  lines  of 
A  with  input  lines  of  B. 

Define  a  relational  system  as  any  set  of  multiple  index  ma¬ 
trices  with  some  common  index  range  K.  called  the  carrier  and  with 
the  following  two  properties: 
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(1)  is  closed  under  matrix  multiplication 

(2)  ft  is  a  subset  of  $  . 

It  is  possible  to  show  that  matrix  multiplication  is  associative  so  cff 
is  a  semigroup.  Also,  /V  is  a  group  whose  unit  element  is  the  scalar 
1  which  is  a  unit  element  for  J?  .  Thus,  is  a  monoid  or  semi¬ 
group  with  a  unit. 

Using  the  definitions  given  earlier  we  may  prove  the  commuta- 

n'  n  n'  n 

tion  law:  A?r  .  A'tt  ,  =  ir  ,  A'tt  ,  A.  A  pictorial  represen- 
-  q+p  q+q  P+P  P+q 

tation  of  the  left  and  right  sides  o i  this  equation  is  shown  on  the  follow¬ 
ing  page. 

The  direct  product  A  (x)  A'  is  defined  to  be  the  quantity  appear¬ 
ing  on  either  side  of  this  equation.  An  alternative  way  of  representing 
A  (x)  A*  is  by  the  diagram  shown  below. 


It  is  possible  to  prove  that  direct  product  is  associative  and  that  it  has 
various  other  properties. 

An  important  special  case  of  a  relational  system  is  that  of  an 
algebra .  Each  matrix  R  in  is  then  single  valued  in  the  sense 


that  for  every  sequence  i(l). . . . ,  i(p)  in  K,  there  is  exactly  one  sequence 
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j(l), . . . ,  j(q)  in  K  such  that  [i(l)  ...  i(p)  rj(l)  . . .  j(q)]  =  1.  All  other 
matrix  elements  have  the  valu'e  0.  Furthermore,  (f\  can  be  generated 
from  and  a  set  J%  Q  of  matrices  with  q  =1. 

For  example,  a  semigroup  may  be  generated  from  a  single  bi¬ 
nary  operation.  Letting  B  denote  this  operation,  we  see  that  p  =  2 

2 

and  q  =  1.  The  associative  law  may  be  written:  BB  =  B  B.  To 
describe  a  monoid  we  include  the  unit  element  E  as  an  operation  with 
p  =  0  and  q  =  1.  The  unit  rules  are  EB  =  Et^B  -  e.  Here,  e  denotes  the 
unit  of  .V  rather  than  the  unit  of  the  monoid  being  described. 

2.6  Relational  Semigroups 

A  relational  semigroup  (RSG)  is  a  slightly  more  abstract  entity 
than  a  relational  system.  It  is  defined  as  any  semigroup  J  with  the 
following  thr-e  properties. 

(1)  contains  the  subgroup  ^  consisting  of  all  finite  permutations 

J 

of  the  natural  numbers.  Also,  the  unit  e  of  y*  is  the  unit  of  a  . 

(2)  Associated  with  each  member  T  of  J  there  are  two  integers  p(T), 
q(T),  both  >  0,  which  obey  the  following  rules. 

(a)  pOT’)  <  max  (p(T).  p(T')  -  q(T)  +  p(T)) 

(b)  q(TT')  <  max  (q(T’),  q(T)  -  p(T’)  +  q(T'); 

(c)  p(TT’)  -  q(TT')  *  p(T)  -  q(T)  +  p(T’)  -  q(T’) 

(d)  p<e)  =  q(e)  =  0.  and  p(S)  =  n, 

where  S  is  any  element  of  J  and  n  is  the  largest  number  not  fixed  un¬ 
der  the  permutation  S.  Note  that  (a)  and  (b)  are  equivalent  because  of 
(c). 
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(3)  The  commutation  law  holds.  In  a  formal  statement  of  this  law,  we 
should  replace  each  of  the  elements  of  %  by  a  corresponding  element 
of  A  . 

It  is  possible  to  show  that  to  every  relational  system  there 
corresponds  an  RSG  defined  in  the  obvious  way  with  each  matrix  of 
the  relational  system  corresponding  to  an  element  of  the  RSG.  It  is 
an  open  question  whether  to  each  RSG  there  corresponds  a  relational 
system. 

Let  J  and  S  '  be  two  RSG’s.  Then  any  mapping  h:  J  -  J  ' 
with  the  following  three  properties  will  be  called  an  RSG  homomorphism. 

(1)  h  preserves  the  product  operation,  i.  e.  for  T^  and  Tg  in  , 
h(Tj  T2)  =  htTjWTj). 

(2)  h  is  the  identity  map  from  A  in  J  to  J  in  J\ 

(3)  a.  p(T)  >  p(h(T)), 

b.  q(T)  >  q(h(T)), 

c.  p(T)  -  q(T)  =  p(h(T))  -  q(h(T)),  for  T  in  J  .  Again, 
a  and  b  are  equivalent  because  of  c. 

If  there  is  a  homomorphism  h  from  an  RSG  J  onto  the  RSG  of  a 
relational  system  S  ,  then  we  shall  say  that  J  represents  Jx  . 
Each  T  in  J  will  be  said  to  represent  its  image  h(T)  in  tf  . 

The  triple  (h.  J  ,  ,\  )  may  be  regarded  as  analogous  to  a  ma¬ 
chine  in  ordinary  automata  theory.  Here,  an  element  T  of  J  corres¬ 
ponds  to  an  input  sequence.  Its  image  H(T)  is  the  Boolean  matrix  des¬ 
cribing  the  set  of  all  pairs  (i.  j)  such  that  T  causes  the  transition  from 
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state  i  to  state  j.  There  are  also  elements  of  J  whose  images  are  a 
row  vector  and  a  column  vector  corresponding  to  the  sets  of  initial  and 
terminal  states  respectively. 

It  may  be  noted  that  an  RSG  plays  the  same  role  with  respect  to 
a  relational  system  as  does  a  theory  to  an  algebra  in  the  Eilenberg  and 
Wright  development  [7]. 

A  subset  T  of  an  RSG  J  will  be  called  recognizable  if  and  only 
if  J  represents  a  relational  system  v\  with  finite  carrier  K  such 
that  T  represents  just  the  true  scalars  in  .  We  may  show  that  this 
type  of  set  is  a  generalization  of  a  regular  set  in  the  automaton  case. 
Similar  closure  properties  may  be  proved  to  hold  but  the  proofs  are 
more  difficult  and  not  simple  generalizations  oi  the  proofs  in  the  auto¬ 
maton  case. 

Let  <A'  be  a  relational  system  with  c  .rrier  K  and  f  any  mapping 
from  K  onto  some  set  K'  .  Then,  for  each  matrix  R  in  <A  we  may 
form  a  matrix  R'  on  the  carrier  K’  letting  [i'(l)  •  • .  i'(p)r'j'U)  . . .  j ’(q) J 
S[i(l)  . . .  i(p)  rj(l)  . . .  j(q)],  where  the  sum  is  taken  over  all  matrix 
elements  of  R  such  that  f(i(l))  =  i'(l), . . . ,  f(i(p))  =  i’(p),  f(j(l))  = 
j’(l), . . . ,  f(j(q))  =  y(q).  If  the  image  of  is  a  relational  sys¬ 
tem  and  if  the  corresponding  mapping  is  an  RSG  homomorphism,  then 
we  shall  call  f  a  relational  system  homomorphism  from  to  ft  '. 

A  somewhat  different  definition  of  homomorphism  has  been  given 
by  Yeh  [4b)  for  relational  systems  generated  by  (1, 1)  index  matrices. 
His  definition  is  more  general  in  some  respects  and  less  general  in 
others  but  fails  to  possess  input-output  symmetry. 
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Again,  if  we  are  given  a  relational  system  with  carrier  K, 
then  we  define  a  subset  G  of  K  to  be  recognizable  if  and  only  if  there 
is  a  relational  system  homomorphism  f  from  f\  to  a  relational  sys¬ 
tem  </? '  with  finite  carrier  K'  such  that  G  consists  of  just  those  ele¬ 
ments  mapping  to  a  given  subc?t  G'  of  K*. 

In  the  case  in  which  ft  is  an  algebra  the  above  definition  may 
be  shown  to  be  equivalent  to  that  given  by  Mezei  and  Wright  [31].  Clo¬ 
sure  results  continue  to  hold,  however,  in  this  generalized  case. 

An  important  and  interesting  question  concerns  when  the  homomorphic 
image  of  a  recognizable  set  is  also  recognizable.  A  partial  answer  to 
this  question  has  been  found. 

2. 7  Free  Relational  Semigroups 

Let  ft  be  a  set  of  letters  w  with  a  pair  (p(w),  q(w)  of  nonnegative 
integers  associated  with  each  w  in  ft.  We  may  then  form  the  free  RSG 
generated  by  ft  which  we  write  ^(ft).  It  has  the  property  that  if  h  is 
any  mapping  from  ft  to  an  RSG  such  that  p(w)  >  p(h(w)), 
q(w)  >  q(h(w)),  and  p(w)  -  q(w)  =  p(h(w))  -  q(h(w))  then  h  may  be  ex¬ 
tended  to  an  RSG  homomorphism. 

One  may  interpret  each  element  T  of  ^/(ft)  as  a  combinational 
network  composed  of  blocks  with  labels  w  taken  from  ft  and  having  p(w) 
inputs  and  q(w)  outputs.  The  network  itself  has  p(T)  inputs  and  q(T) 
outputs.  Each  input  of  a  block  is  either  an  input  of  the  network  or  else 
connected  to  an  output  of  one  other  block.  A  similar  rule  holds  for  out¬ 
puts.  If  j£(ft)  represents  a  relational  system  A  ,  then  for  any  T  in 


62 


97  (ft)  we  may  regard  the  image  h(T)  in  as  the  operation  carried 
out  by  the  network  T. 

Another  application  of  ^(ft)  is  to  language  theory.  T,et  ft  con¬ 
sist  of  a  set  ax  of  (0, 1)  symbols  A,  a  single  (2, 1)  symbol  B,  and  a 
(1,  0)  symbol  C.  Then,  we  may  use  g^(ft)  to  represent  the  free  semi¬ 
group  tJ  (tTc  )  generated  by  JiT  .  Each  ktuX  represents  a  row  vec¬ 
tor  with  a  single  unit  corresponding  to  that  generator  in  {<Jf  ),  the 
element  B  represents  the  concatenation  operation,  and  C  represents 
a  column  vector  consisting  only  of  units.  Since  ^(i ft  )  is  an  algebra, 
each  row  vec.  ***•  contains  a  single  unit  and  represents  the  element  or 
elements  O’-  j'L  )  corresponding  to  the  position  of  that  unit. 

if  C  is  recognizable  on  f/»(ft ),  then  the  row  vectors  R  such  that 
RC  is  represented  by  a  member  of  T  are  just  the  strings  in  a  context 
free  language.  Conversely,  for  every  context  free  language  on  {<J() 
there  is  a  recognizable  T  on  9 '(ft).  The  finite  relational  system 
which  is  used  to  recognize  T  is  just  the  context  free  grammar  of  the 
language.  Productions  in  this  grammar  constitute  a  list  of  the  unit 
(or  true)  matrix  elements.  The  matrix  element  [ i(l)i(2)bj(  1) J  =  1  of 
B  is  written  as  a  production  j(l)  **  i(l)i(2),  and  the  matrix  element 
[a j(l) ]  =  1  of  A  €  <JZ  is  written  j(l)  -  A  where  A  is  taken  as  a  terminal 
symbol.  Finally  (i(l)cj  s  1  of  C  is  written  C  -  i(l),  where  C  is  the 
initial  symbol.  This  difference  in  convention  is  of  little  consequence. 
However,  we  may  note  that  the  elements  of  T  may  be  regarded  as 
trees,  or  bracketed  strings  white  their  images  in  4  (uT  )  nave  had 


63 


the  brackets  removed. 

Another  interesting  avenue  of  study  concerns  relational  systems 
in  which  the  group  •#  is  augmented  by  certain  other  matrices  and  the 
closure  is  taken  under  matrix  multiplication.  One  such  matrix  is  the 
column  vector  consisting  entirely  of  units,  already  mentioned  in  con¬ 
nection  with  languages.  Another  is  the  bifurcator  or  (r, 2)  index  ma¬ 
trix  F  in  which  [i(l)fj(l)j(2)j  is  true  if  and  only  if  i(l)  =  j(l)  =  j(2). 

In  the  system  of  Eilenberg  and  Wright  [7]  both  these  operations  are 
implicitly  assumed.  Finally,  a  third  matrix  is  one  which  may  be 
thought  of  as  a  union.  This  is  a  (2, 1)  index  U  in  which  [ i(l)i(2)uj(l)] 
is  true  if  and  only  if  either  i(l)  =  j(l)  or  i(2)  =  j(l). 


Section  3 


Asymptotic  Decomposability  of  Machines 
3. 1  Introduction 

Series-parallel  decomposition  of  sequential  machines  has  been 
studied  extensively  from  the  point  of  view  of  the  decomposition  of 
specific  machines  [18, 23,  47].  Putzolu  [36,  37]  has  recently  obtained 
results  concerning  the  likelihood  that  randomly  chosen  machines  ad¬ 
mit  this  decomposition.  His  analysis  treated  the  case  in  which  a 
state  behavior  type  of  realization  was  assumed  and  decomposition  in¬ 
volving  state  splitting  was  not  permitted.  In  the  present  paper  we  at¬ 
tack  the  analogous  problem  in  which  state  splitting  is  allowed. 

For  definiteness  and  convenience  we  define  here  some  of  the 
concepts  which  will  be  needed  in  our  study.  Let  j[n]]  represent  the  set 
{l, . . . ,  n} ,  when  n  is  any  positive  integer.  Define  a  machine  with  n 
states  and  p  inputs  as  any  mapping  6  from  In]]  x  [[pj  to  In]],  and  write 
jjn]jInE  x  IpE  t0  represent  the  set  of  all  such  mappings.  Following 
[18],  a  decomposition  of  6  e  [nE^n^  X  into  two  machines 

Injlf*  IpI  In2!  *  ttnjpB 

6  j  c  InjE  and  6  2  e  In2E  is  specified  >y  giving  a 

mapping  h  from  a  subset  of  In^E  x  I1^!  onto  1^1  such  that  whenever 
i  =  h(j.  k)  and  x  €  Ipl,  then  h(5  ^(j,  x),  6  2(k,  (j~l)p+x))  is  defined  and 
equals  6  (i,  x).  A  decomposition  of  6  into  more  than  two  machines  is 
defined  and  equals  o  (i,  x).  A  decomposition  of  6  into  more  than  two 
machines  is  defined  recursively  as  the  result  of  further  decomposing 
either  5^  or  $2  or  both  and  repeating  an  arbitrary  number  of  times.  An 
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r-component  decomposition  is  one  which  results  from  the  performance 
of  the  basic  decomposition  process  r-1  times  and  therefoie  yields  r 
machines.  We  need  not  formulate  here  the  well  known  associative 
principle  which  permits  one  to  speak  of  the  decomposition  of  6  into  r 
machines  5  ,  6 r  without  specifying  the  steps  by  which  they  were 

obtained.  We  shall  say  that  5  is  non-trivially  decomposable  if  it 
admits  an  r-component  decomposition  in  which  each  of  the  resulting 
r  machines  has  fewer  states  than  6  . 

If  one  is  interested  only  in  state  behavior  realizations,  then  the 
mapping  h  is  always  taken  as  one-to-one.  In  this  case  to  determine 
whether  or  not  a  me  'line  is  nontrivially  decomposable  one  need  only 
consider  2-  component  ueoompositions.  However,  in  the  more  general 
c:  se  treated  here  there  is  no  such  restriction. 

3,  2  A  Criterion  for  Decomposability 

A  machine  6  €  fn]]^n^  x  is  called  a  group  machine  if  each 
restriction  6  (  .  x)  is  a  permutation  of  [n]j.  From  any  machine  6 
we  may  derive  a  group  machine  6  ’  by  the  simple  expedient  of  letting 
6  ’(  ,  x)  =  6  (  ,  x)  if  6  {  ,  x)  happens  to  be  a  permutation  and  letting 
6  \  .  x)  be  the  identity  map  otherwise.  We  note  that  the  corresponding 
group  machine  is  the  same  as  the  original  machine  if  and  only  if  the 
original  machine  is  a  group  machine. 

Theorem  1 

A  machine  5  is  nontrivially  decomposable  if  and 
only  if  the  corresponding  group  machine  6  ’  is 
nontrivally  decomposable. 
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This  theorem  may  be  easily  proved  using  results  of  Krohn  and 
Rhodes  [23]  or  of  Zeiger  [47].  Therefore  we  do  not  present  a  formal 
proof  here.  In  following  the  construction  of  Zeiger  one  may  decom¬ 
pose  the  machine  6  into  components  6  ^  and  6  ^  where  6  g  has  fewer 
states  than  6  ,  and  £  ^  is  a  permutation  reset  machine  of  the  same  num¬ 
ber  of  states  as  5  .  The  permutations  of  6  ^  correspond  to  those  of 

5  ’  and  one  may  then  show  that  to  any  nontrivial  decomposition  of  6  ’ 
there  is  an  isomorphic  nontrivial  decomposition  of  6  .. 

As  a  corollary  to  theorem  1  we  have  the  result  that  if 

6  e  [[n]j^n^  x  is  a  machine  of  n  >  3  states  such  that  for  no  input 
x  e  Ipl  is  6  (  ,  x)  a  permutation,  then  6  is  nontrivially  decomposable. 

It  is  this  corollary  which  is  required  in  the  proof  of  theorem  2  in  the 
next  section.  We  note  that  in  Zeiger’s  construction,  if  £  (  ,  x)  is  never 
a  permutation,  then  6  ^  is  a  reset  machine.  Then,  any  nontrivial 
partition  on  [[ n]]  ~  [[n^]]  has  the  substitution  property  for  6  .  Hence 

5  1  (and  consequently  6  )  is  nontrivially  decomposable. 

3.3  Decomposability  of  Randomly  Selected  Machines 

In  this  section  we  shall  be  concerned  with  estimating  what  frac¬ 
tion  of  the  set  ([n]]^n^  x  is  pontxivially  decomposable,  and  in  par¬ 
ticular  shall  investigate  the  way  this  function  of  n  and  p  behaves  as  n 
and  p  approach  <©  . 


Theorem  2 

If  pn^e  n-Oasn-*oo,  then  the  probability  ap¬ 
proaches  1  that  a  machine  6  will  be  nontrivially 
decomposable  if  it  is  chosen  at  random  from 
[njM  x  M 

Proof :  If  6  e  []  nj]^n^  *  ^  is  a  machine  such  that  for  no  x  €  dp]] 
is  6  (  ,  x)  a  permutation  then  6  is  surely  decomposable  by  the  fol¬ 
lowing  reasoning.  Since  n  >  2  and  6  '(  ,  x)  is  the  identity  function 
for  all  x,  we  may  decompose  6  '  in  any  way  we  wish  using  state  be¬ 
havior  realizations  and  identity  functions  for  the  components.  By 
theorem  1,  5  is  thus  nontrivially  decomposable.  Now  there  are  n! 
permutations  and  nn  mappings  of  |n]|  into  j[nfl  so  the  probability  that 
no  mapping  6  (  ,  x)  be  a  permutation  is  (1  -  -^) .  However  using 

Stirling's  formula,  we  have  (1  -  ~)p  >  1  -  =  1  -  \[%v  pn^e-11 

j  n  ~  n 

(1  +  0(-))  -  1  as  r  *  oo.  This  completes  the  proof  of  the  theorem. 

Lemma  1 

The  fraction  of  all  permutations  of  n  letters  in  which 

[n/m] 

no  cycle  has  a  length  divisible  fcy  m  is  n  (1  -  -r— ). 

i_l  lm 

Proof :  Write  Tm(n)  for  the  number  of  permutations  o  of  [[n]]  having 
no  cycle  with  a  length  divisible  by  m.  For  r  <  m,  if  the  cycle  con¬ 
taining  the  letter  1  has  length  r,  then  there  are  Tm(n-r)  ways  in  which 
the  remaining  cycles  may  be  chosen.  The  cycle  containing  1  may  be 
chosen  in  (n- 1 ) !  /(n- r) !  ways.  Hence,  there  are 
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m-1 

Yj  [(n- 1)  1  /(n-r)!]T  (n-r)  permutations  of  type  a  in  which  the  cycle 
r=l  m 

containing  1  has  length  less  than  a.  Now,  the  cycle  containing  1  must 
not  have  length  m,  but  if  it  lias  length  greater  than  m,  then  we  may 
treat  the  portion  of  this  cycle  consisting  of  1  and  the  chain  of  m  let¬ 
ters  following  1  as  if  they  were  a  single  letter.  This  chain  may  be 
chosen  in  (n-l)l /(n-m-1)!  ways  and  following  its  choice  there  are 
Tm(n-m)  ways  in  which  the  remainder  of  the  permutation  may  be  cho¬ 
sen..  Thus,  we  obtain 
m-1 

Tm(n)  =  ^  [(n-l)!/(n-r)!]Tm(n-r)  +[(n-l)!/(n-m-l)!]Tm(n-m). 

r=l 

This  difference  equation  has  a  unique  solution  subject  to  the  conditions 

that  T  (11  =  1  and  T  (0)  =  1,  where  it  is  assumed  that  coefficients  for 
m  '  m'  ' 

terms  in  T  (n-r)  vanish  when  r  >  n.  To  prove  the  lemma,  we  show  that 
m 


1 

T  (n)  =  n!  II  (1  -  -r— )  is  a  solution  to  the  difference  equation . 
m  lm 

We  may  check  directly  that  Tm(n)  =  n!  when  n  <  m  and  also 
Tm(m)  =  (m-l)(m-l)!.  To  justify  our  formula  when  n  >  m  it  is 
more  convenient  to  derive  the  following  (m+l)-st  order  difference 
equation : 

nTm(n-1)-Tm(n)  =  [(n-l)!/(n-m)!](n-m-l)[(n-m)Tm(n-m-l)-Tm(n-m)]. 

[-1 

m  1 

Substituting  Tm(n)  =  n!  n  (1  -  j^),  we  may  check  that  both  sides 


vanish  when  n  is  not  divisible  by  m.  Otherwise  both  sides  have  the 
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C£) 

m  ^ 

value  (n-1)!  n  (1  -  -t—).  No  spurious  solution  is  possible  since 
i=j  lm 

the  first  m+1  points  agree  with  the  solution  to  our  original  equation. 
This  completes  the  proof  of  the  lemma. 

An  immediate  consequence  of  this  lemma  follows  from  the  fact 
that  when  m  is  a  prime  or  a  power  of  a  prime,  the  order  of  a  permu¬ 
tation  is  divisible  by  m  if  and  only  if  it  has  a  cycle  vhose  length  is 
divisible  by  m.  In  this  case,  there  are  exactly 


Tm(n)  =  n!  n  (1  -  permutations  of  n  letters  whose  order  is  not 

divisible  by  m.  In  our  application  the  exact  formula  is  less  convenient 
than  an  asymptotic  expression  which  may  be  derived  as  follows. 


n  (1  -  ~) 

i=l  im 


n  (i  -  --) 
1=1 

[— ]! 

1  nr 


[-]  m 
1  m 1 

T(1  -4) 
m 


[-1  m 

1  m  1 

T“ 

rd  ' 

m 


The  asymptotic  result  holds  when  m  remains  fixed  and  n  -oo,  see 

1 

reference  [43].  Also,  n!  ~  VE?  n^  (^)n,  so  we  have 

rr  _  mm  7JF  “  m\n,n 
T  (n)  ~  - r  n  (-)  . 

T(1  -  i  )  e 
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In  particular,  we  obtain: 


1 

T2(n)  ~  2(|)n,  and  Tg (n)  ~  2.67n^  (^)n. 


The  second  of  the  v  r'f/.ressions  is  used  in  the  proof  of  theorem  3. 

A  group  G  of  permutations  on  a  set  [[n]]  of  n  letters  is  called 
imprlmitive  [16]  if  there  is  a  nontrivial  partition  on  Jn]]  into  disjoint 
subsets  Sj, . . . ,  sf  with  the  substitution  property  [18].  This  means  that 
for  all  g  €  G  and  sets  s i  in  the  partition,  there  is  some  set  in  the 
partition  such  that  s.g  s..  Since  G  is  a  group,  one  can  easily  show 
that  actually  s^  -  Sj.  If  there  is  no  such  partition,  then  G  is  called 
primitive  and  we  have  the  following  lemma. 

Lemma  2 

Two  randomly  chosen  permutations  on  a  set  of  n 
letters  generate  a  primitive  group  with  probability 
approaching  1  as  n  approaches  oo. 

Proof ;  Again  write  [[ n]j  =  { 1 , . . . ,  n)  for  the  set  of  n  letters  to  be 
permuted,  and  let  a  and  y  be  chosen  randomly  from  the  sei  of  n!  per¬ 
mutations  of  [[ n]] .  We  see  that  the  group  generated  by  a  and  y  is  im- 
primitive  if  JnT;  ^an  be  partitioned  into  two  nonempty  sets  of  cardi¬ 
nality  n^,  and  n-n^  such  that  a  andy  both  permute  the  letters  of  these 

sets  among  themselves.  Partitions  of  this  type  will  be  called  separa- 

2  2 

ble,  and  we  see  that  there  are  (nj!)  ((n-n^) ! )  ways  in  which  a  ar.d  y 
may  be  chosen  so  that  the  partition  into  two  such  sets  is  separable. 
While  it  is  clear  that  for  a  given  a  and  y  there  may  be  more  than  one 
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separable  partition,  we  have  the  upper  bound 


number  of  ways  a  and  y  may  be  chosen  so  as  to  allow  [[ nj  to 

2 

have  a  separable  partition.  Thus,  of  the  (n!)  ways  to  choose  a  and 
y  the  fraction  allowing  a  separable  partition  is  no  greater  than 


r-i 

L2  n  ifn-m)! 
y  1  1 ' 

L 

npl 


n! 


1  2 

The  first  two  terms  of  this  sum  are  -  +  —> — r-y 

n  n(n-l) 


n  6 

while  the  [^]  -  2  remaining  terms  are  each  no  greater  than  n(n  • 


[?i 


f  njKn-nj)! 

We  may  therefore  write  - =  -  +  O  {—%),  and  we  ob- 

n^=l  ’  n 


serve  that  this  bound  approaches  0  as  n  approaches  oo. 

Even  if  there  is  no  separable  partition  of  || nj]  there  may  yet  be 

a  partition  with  the  substitution  property.  For  this  to  occur,  each 

class  in  such  a  partition  must  be  capable  of  being  mapped  into  any 

given  class  by  a  suitably  chosen  permutation  in  the  group  generated  by 

o  andy  .  Hence,  all  classes  must  contain  the  same  number  of  letters. 

Let  be  the  number  of  letters  in  a  class  and  a  the  number  of  classes. 

Then  n  =  a  n.  where  1  : a.  n,  and  there  are  — -  ways  ot  form- 

1  1  a!(n  !) 

ing  such  a  partition.  For  any  given  partition  oi  this  type  the  permutations 
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cL  m 

a  and  y  may  together  be  chosen  In  [a!(n^ !)  ]  ways  so  that  the  sub- 

£ 

stitution  property  is  satisfied.  Thu  n!  a!  (n^!)  is  an  upper  bound 

to  the  number  of  ways  that  o  and  y  may  be  chosen  so  some  partition 

into  a  equal  classes  has  the  substitution  property.  The  ratio  of  this 
2 

bound  to  (n!)  may  be  written 


airarrii  =  •  •  • 


n  n  n 


We  have  taken  a  >  1  and  as  a  consequence  no  factor  in  this  product 

2 

may  be  greater  than  ^  and  an  upper  bound  to  the  product  is  there- 
2  n-a  2  $ 

fore  (j)  <  (j)  .  Since  the  number  of  possible  (jhoices  for  n^ 

2W 

clearly  cannot  exceed  n,  there  is  the  upper  bound  n(^)  to  the 

K.' 

probability  that  for  randomly  chosen  permutations  a  andy  there  is 
a  partition  with  the  substitution  property  which  is  not  separable.  Since 
this  bound  also  approaches  0  as  n  approaches  00  the  proof  of  the  lem¬ 
ma  is  complete. 

One  may  conclude  slightly  more  than  is  contained  in  the  state¬ 
ment  of  the  lemma  if  a  trivial  refinement  is  made  in  the  proof.  The 
actual  fraction  of  the  pairs  a,y  having  a  separable  partition  with 

n,  =1  is  less  than  -  but  is  asymptotic  to  -  as  n  -00.  Thus,  we  see 
1  n  r  n 

that  these  partitions  account  for  almost  all  cases  in  which  the  group 


generated  by  a  a*- 


rmpromitive  asn-x.  The  prooability  that 
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this  group  will  be  imprimitive  is  thus  asymptotic  to  -  . 

Theorem  3 

1 

If  pnVn  1  -  qo  as  n  -  qo,  then  the  probability  approaches 
0  that  a  machine  6  will  be  nontrivially  decomposable  if 
it  is  chosen  at  random  from  ([n]j^ri^  X 
Proof :  By  the  theory  of  Krohn  and  Rhodes  [23],  if  there  exist  inputs 
x  e  dp]]  such  that  the  corresponding  mappings  6(  ,x)  are  permutations 
which  generate  the  alternating  group  An  on  n^4  letters,  then  the  ma¬ 
chine  6  is  indecomposable.  This  fact  follows  because  some  compo¬ 
nent  in  any  decomposition  of  5  must  have  an  input  semigroup  with  a 
subgroup  which  is  the  inverse  homomorphic  image  of  An»  Therefore, 
this  component  must  have  at  least  n  states,  so  6  fails  to  satisfy  our 
criterion  of  nontrivial  decomposability.  We  shall  show  that  the  prob¬ 
ability  approaches  1  as  n  approaches  oo  that  A^  is  a  subgroup  of  the  in¬ 
put  semigroup  of  6  . 

Let  s  be  any  permutation  of  [[n]j  which  cyclically  permutes  some 
ret  { i,  j.k}  C_  H n]]  of  three  states  and  produces  a  permutation  of  some 
order  v  on  the  remaining  states  which  is  not  divisible  by  3.  Then  s 
may  be  used  to  generate  a  cyclic  permutation  t  of  { i,  j,  k},  since  we  may 
take  t=sI/  which  cyclically  permutes  {i,  j.k},  leaving  the  remaining  let¬ 
ters  fixed.  We  compute  that  there  are  cyclic  permutations 

of  some  set  {i,  j,k]  C_  Jnfl.  Furthermore,  for  each  such  permutation 
there  are  T^(n-3)  permutations  of  the  remaining  states  whose  order  v 
is  not  divisible  by  3.  Hence,  there  are  i--n  — -  T„(n-3)  permutations 
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Hence,  the  probability  approaches  1  that  at  least  one  of  the  inputs 
x  e  [[  [P]  J  yields  a  permutation  s  =  6  (  ,  x)  which  may  be  used  to  gen¬ 
erate  a  cyclic  permutation  t  of  three  s  ites. 

The  probability  also  approaches  1  that  each  of  the  two  sets 
[[  [^]  ]1  -  II  [fl  S  and  f  pi  -  I  [^j  |  contains  some  input  yielding  a 
permutation  because  these  sets  contain  on  the  order  of  |  inputs.  By 
lemma  2,  we  note  that  two  randomly  chosen  permutations  of  ([ n[j  gen¬ 
erate  a  primitive  group  G  with  probability  approaching  1  as  n  approaches 
qo.  Hence,  we  assume  that  a  primitive  group  G  on  J n||  is  generated  from 
two  ot  the  inputs  in  dpfl  -  |  (^1  1  and  that  a  cyclic  permutation  t  oi  three 
states  is  generated  from  one  of  the  inputs  in  f  [*]  1- 
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cyclically  permute  three  states,  leaving  all  others  fixed.  It  is  known 
[16]  that  if  contains  all  such  cyclic  permutations,  then  An  is  a  sub¬ 
group  of  G  u  { t} .  Define  0  £_  j[n]]  x  fnD  as  the  set  of  all  pairs  (it,  i2) 
both  lying  in  some  cycle  of  an  element  of  V"  •  Then  9  is  an  equivalence 
relation  since  if  (ij,  i2)  lie  on  a  cycle  of  tj  and  (ig,  ig)  lie  on  a  cycle  of 
t2,  then  (ij,  i 3)  lie  on  a  cycle  i2  ig)  =  tj  t'1  t~*t2  if  the  third  mem¬ 
bers  of  the  cycles  for  (i^,  i2)  and  (ig,  ig)  are  distinct  and  on  a  cycle 
(Lijig)  =  tgtj  if  die  third  members  are  the  same.  This  construction 
shows  us  thai  whenever  i  ,  ig,  and  ig  are  in  the  same  9  class,  then 
the  cycle  (ijigig)  is  in  IJ.  .  Also,  9  induces  a  partition  o»i  [[nfl  which 
has  the  substitution  property  with  respect  to  G.  For,  let  g  be  any  ele¬ 
ment  of  G,  then  (i^,  i2)  €  6  with  corresponding  tj  e  ij-  ,  we  see  that 
g  *tjg  is  also  a  cyclic  permutation  in  g  for  the  pair  (ijg,  i2g).  Conse¬ 
quently  (ijg,  igg)  is  in  9.  But  G  was  assumed  to  be  primitive  and 
is  nonempty  since  it  contains  t,  so  must  contain  all  cyclic  permu- 
tations  of  three  states.  Thus,  y,  generates  A  ,  with  probability 
approaching  1  as  n  approaches  oo.  This  completes  the  proof. 

3.  4  Additional  Remarks 

The  condition  for  nontrivial  decomposability  stated  in  section  3. 1 
applies  to  each  input  of  a  machine  separately.  Therefore,  for  fixed 
n,  the  probability  of  nontrivial  decomposability  can  never  increase  as 
p  increases.  However,  the  theorems  of  section  3.3  fail  to  provide  in¬ 
formation  concerning  the  probability  of  nontrivial  decomposability  in 
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certain  cases  as  indicated  by  the  following  corollary  to  theorems  2 
and  3. 

Corollary  1 

If  n^~nn~  approaches  a  limit  $  as  n  approaches  oo  then 

the  probability  of  nontrivial  decomposability  approaches 

1  if  f  >  2  and  approaches  0  if  C  < 

Proof:  If  iim  ( |"-~>)  >  then  for  some  e  >  0  and  n^  we  have 
~  ~  n-oo  j 

"liTh"**  >  ^  +  e  whenever  n  >  nQ.  Hence,  p  n^  e"n  <  l/ne  and  since 

i\(me  -  0  as  n  -  oo,  we  may  apply  theorem  2.  If  lim  ^ 

n-oo 

then  for  some  e  >  0  and  n^  we  have  ^  -  e  whenever 

1 

n  >  nQ.  Hence,  p  n  e"  >  ne  and  since  n  -oo  as  n  -  oo,  we  may 
apply  theorem  3.  This  completes  the  proof. 

We  note  that  nothing  is  known  about  the  case  in  which  f  lies  in 

i  1 

the  range  ^  <  t  <  It  is  conjectured  that  this  "gap"  is  removable 

and  that  a  more  delicate  pair  of  theorems  similar  to  2  and  3  is  valid 

in  which  the  same  exponents  of  n  appear.  In  particular,  it  seems 

1 

5  -n 

possible  that  the  condition  p  n  e  -oo  may  be  sufficient  to  obtain 
the  conclusion  of  theorem  3. 


Corollary  2 


approaches  a  limit  £ '  as  n  approaches  oo,  then 


the  probability  of  nontrivial  decomposability  approaches 
1  if  ( ’  >  1  and  approaches  0  if  f '  <  1. 
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This  result  follows  easily  from  corollary  1,  and  permits  one  to 
compare  the  probability  of  this  type  of  decomposition  with  state  be¬ 
havior  decomposition.  In  [36,  37]  it  is  shown  that  the  limiting  value 
In  n 

of  — —  is  an  appropriate  quantity  to  consider  in  order  to  determine 
whether  the  limiting  probability  of  nontrivial  state  behavior  decompo¬ 
sition  is  1  or  0. 

Another  question  of  some  interest  concerns  whether  or  not  a  ma¬ 
chine  6  is  nontrivially  decomposable  into  just  two  components  6  ^  and 
6  2>  each  having  fewer  states  than  6  .  As  was  pointed  out  in  the  intro¬ 
duction,  a  machine  may  be  nontrivially  decomposable  without  being 
nontrivially  decomposable  into  two  components.  An  example  is  the 
universal  4-state  machine  which  is  decomposable  into  4  components 
of  3,  2, 2  and  3  states,  but  not  into  two  components  each  of  fewer  than 
4  states. 

According  to  [lu]  a  machine  admits  a  nontrivial  2-component 
decomposition  if  and  only  if  it  admits  a  nontrivial  set  system  decom¬ 
position. 

Using  methods  similar  to  those  used  in  the  proof  of  theorem  2, 
it  is  possible  to  show  that  if  ^  as  n  -  oo,  then  for  any  k  >  0, 

the  probability  that  a  randomly  chosen  5  c  X  admits  at 

least  k  nontrivial  SP  set  systems  approaches  1;  correspondingly,  the 
expected  number  of  nontrivial  SP  set  systems  of  a  machine  approaches 


00. 
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An  unsettled  question  is  whether  set  system  (i.  e.  2-component) 
nontrivial  decomposability  is  Almost  surely  satisfied  if 
\  -  j  as  n  “,°°*  K  is  hoped  that  further  studies  may  pro¬ 

vide  answers  to  some  of  these  questions. 


Section  4 


The  Recognizability  of  Equational  Sets 

Automata  theory  and  the  theory  of  formal  languages  are  closely 
related:  automata  recognize,  parse  and  translate  languages;  languages 
are  used  to  program  and  describe  automata.  This  research  is  con¬ 
cerned  with  some  specific  results  relating  the  generation  and  recog¬ 
nizability  of  languages. 

It  is  well  known  to  automata  theorists  that  a  language  is  gene¬ 
rated  by  a  left  linear  grammar  iff  it  is  recognized  by  some  finite  au¬ 
tomaton.  In  this  case,  there  is  an  effective  process  whereby  one  can 
find  a  deterministic  system  (i.  e.  finite  automaton)  which  recognizes 
the  language  generated  by  a  non-determ inistic  system  (i.  e.  left 
linear  grammar).  It  has  been  shown  by  Mezei  and  Weight  [31]  that 
this  effective  process  can  be  generalized  to  any  case  where  the  lan¬ 
guages  in  question  are  subsets  of  the  set  T  of  all  fully  parenthesized 
legal  algebraic  expressions  or  computation  trees  on  some  fixed  set 
of  operator  symbols  i7  (each  symbol  in  0  has  a  given  constant  spec¬ 
ifying  the  number  of  operands  required).  In  this  generalization, 
"generated  by  a  left  linear  grammar"  is  replaced  by  "generated  by  a 
context  free  grammar  which  allows  only  legal  algebraic  expressions 
(on  Q  and  some  finite  set  of  nonterminals)  to  be  substituted  for  non¬ 
terminals.  "  (Such  sets  are  called  equational  by  Mezei  and  Wright  [ 31  j , 
grammatical  by  Muller  [33],  and  algebraic  by  Eilenberg  and  Wright 
[ 20] .  They  might  also  be  called  algebraic  context  free  languages. ), 
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Mit  is  recognized  by  some  finite  automaton”  is  replaced  by  "it  is  the 
inverse  homomorphic  image  in  T  of  a  subset  of  a  finite  algebra  on 
S3 .  ”  (Such  sets  are  called  "recognizable”  by  Mezei  and  Wright  [31], 
and  "finite  tree  automaton  definable"  by  Rabin  [38].  Notice  that  T  mi  st 
be  made  into  an  algebra  on  S3  for  homomorphism  to  be  defined;  this 
can  be  done  in  a  natural  way,  and  the  resulting  algebra  is  called  the 
generic  algebra  on  S3 . 

The  definitions  of  equationality  and  recognizability  can,  in  feet, 
be  made  in  any  algebra  OX  on  S3.  Equational  sets  in  Jr  are  precisely 
the  homomorphic  images  of  equational  sets  in  T.  Recognizable  sets 
in  JC  are  the  inverse  homomorphic  images  in  iK  of  subsets  of  fi¬ 
nite  algebras  on  S3 .  It  is  now  of  interest  to  ask  for  which  algebras 
the  equational  sets  and  the  recognizable  sets  coincide. 

In  any  finitely  generated  algebra,  recognizable  sets  are  equa¬ 
tional.  However,  in  a  finitely  generated  free  semigroup  the  equational 
sets  are  the  usual  context  free  languages  while  the  recognizable  sets 
are  just  the  regular  events.  Hence  there  exist  finitely  generated  alge¬ 
bras  in  which  not  all  equate  ,nal  sets  are  recognizable.  Thus  the  fol¬ 
lowing  conditions  on  a  finitely  generated  algebra  ox  on  S3 ,  each  of 
which  by  itself  implies  that  the  equational  sets  of  u7  are  recognizable, 
are  of  interest ; 

1)  oT  Is  isomorphic  to  a  generic  algebra  modulo  a  con¬ 
gruence  Pq  defined  by  an  algebraic  phrase  structure 

*  * 

grammar  G  by  (x,  y)  €  p^  iff  x  =g=>  and  y  =^=>  x.  (See 
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Ginsburg[l5]  for  definitions  of  phrase  structure  grammar 
* 

and  =^f>  ;  when  the  strings  being  operand  are  legal  alge¬ 
braic  expressions  on  ft  and  some  set  of  nonterminals,  G 
is  called  algebraic  iff  the  rules  of  G  allow  only  replace¬ 
ment  of  one  legal  algebraic  subexpression  by  another). 

This  condition  includes  the  case  where  ( Jt  is  finitely  pre¬ 
sentable  on  a  generic  algebra.  The  following  is  used  in 
the  proof  of  sufficiency:  Theorem:  Every  algebraic  phrase 
structure  language  in  a  generic  algebra  is  equational. 

2)  J l  is  isomorphic  to  a  generic  monadic  algebra  modulo  a 
congruence  which  is  a  binary  transduction.  (See  Elgot  and 
Mezei  [8]  for  the  definition  of  binary  transduction. )  This 
includes  the  monadic  case  of  (1)  but  is  not  included  by  it, 
as  may  be  seen  using  a  monadic  algebra  with  idempotent 
operators.  The  latter  is  of  interest  in  asynchronous  switch¬ 
ing  theory.  The  condition  above  has  been  generalized  to 
non-monadiC  algebras. 

3)  j:  is  a  subset  of  an  algebra  &  on  some  set  ft'  of  opera¬ 
tor  symbols,  where  the  equational  subsets  of  are  recog¬ 
nizable,  and  where  certain  relations  hold  between  the  opera¬ 
tions  defined  on  JZ  by  ft  and  the  operations  defined  on  ,S 
by  ft'.  This  result  is  due  to  Muller  [33]. 


If  the  equational  sets  of  an  algebra  01  are  recognizable,  then 
all  word  problems  on  J(  are  solvable.  The  converse,  unfortunately. 
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does  not  hold. 

It  is  thought  that  these  results  have  some  implications  for  paren¬ 
thesis  languages,  loop  free  parallel  computation,  data  structures 
(lists  in  particular)  and  possibly  other  areas. 


Section  5 


Algebraic  Isomorphism  Invariants  for  Transition  Graphs 

The  material  presented  here  summarizes  and  concludes  re¬ 
search  already  appearing  in  the  1966  and  1967  annual  reports.  In 
the  meantime  these  results  were  issued  as  Systems  Engineering 
Laboratory  Technical  Report  No.  21,  "Algebraic  Isomorphism  In¬ 
variants  for  Transition  Graphs, "  where  full  proofs  and  bibliographical 
references  are  to  be  found.  Heie  an  effort  has  been  made  to  present 
in  a  self-contained  and  compact  form  the  more  significant  results. 

A  transition  graph  G  is  a  finite,  directed  graph  such  that  every 
point  of  G  has  outdegree  0  or  1.  (By  a  directed  graph  G  =  (X,y)  we 
mean  the  graph  of  a  relation  y  on  a  set  X.  Thus  "loops"  are  permitted. ) 
Thus  G  =  (X,  y)  is  a  transition  graph  if  and  only  if  y  is  a  partial  trans¬ 
formation  on  X.  Accordingly,  we  say  that  a  transition  graph  G  =  (X,y) 
is  a  transformation  graph  if  y  is  a  (complete)  transformation  on  X 
and  that  G  =  (X,y)  is  a  permutation  graph  if  y  is  a  permutation  on  X. 

The  composition  G  »  G'  of  two  graphs  G  =  (X,y)  and  G’  =  (X, y ') 

(G  and  G’  are  defined  on  the  same  set  of  points)  is  the  result  of  com¬ 
posing  their  lines,  i.  e. 

G  o  G'  =  (X,  yy') 

where  yy'  is  the  composition  of  the  relations  y  andy'. 

Let  A  n  denote  the  set  of  all  digraphs  on  the  points 

Nn  =  {1,2 . n}.  Then  obviously  {.M  n>  •)  is  a  semigroup.  If 

further  we  let  ')>■  n(F)  denote  the  multiplicative  semigroup  of  linear 
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transformations  on  an  n-dimensional  vector  space  V  over  a  field  F, 
the  natural  representation  of  M  relative  to  F  and  some  represen¬ 
tation  has  is  Q-  =  . . .  ,ol}  (CL  is  any  basis  for  V)  is  the 

"  x  ^  n 

function: 

p:  Mr~  A,<F> 

where  p(G)  =  is  defined  as  follows: 

f  1  if  (i,  j)  €  y 
n 

“iTG  =  Vi  Wh6re  a«  *  { 

[  0  otherwise, 

for  all  ai  e  ft.  (Thus,  the  natural  representation  of  relative 
to  F  and  Cl  is  simply  a  linear  transformation  equivalent  of  the  usual 
representation  of  graphs  by  adjacency  matrices.  More  precisely,  if 
the  adjacency  matrix  AQ  of  G  is  regarded  as  being  over  the  represen¬ 
tation  field  F  then  is  just  the  matrix  of  with  respect  to  the  repre¬ 
sentation  basis  Cl  . ) 

In  terms  of  the  natural  representation  we  observe,  first  of  all, 
the  following  well  known  result  (usually  stated  for  adjacency  matrices 
over  the  reals): 

Theorem  1 

If  two  graphs  G  and  G'  are  isomorphic  (G  =  G')  then, 

under  the  natural  representation  (relative  to  any  choice 

of  representation  field  F  and  basis  (1  ),  the  representing 

transformations  T_  and  T_,  are  similar  (T_  ~  T„,). 

u  vj  vj  G 


•  •  •  ■  -iv &*#*&?**  -  ^  * ■  *-■  v* .K#r, ,.;  v* 
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Note  that  the  condition  "T^  ~  T^,,"  is  stronger  than  that  of  being 
"cospectral"  since  the  latter  requires  only  that  T„  and  T„.  have  the 

vjr  li 

same  characteristic  polynomial.  In  particular,  for  example,  one  can 
show  that  all  n-point  directed  trees  which  are  either  to  or  from  a  point 
must  be  cospectral  and  yet  many  such  trees  can  be  datinguished  by  the 
fact  that  their  representations  are  nonsimilar. 

In  what  follows  we  assume  that  the  natural  representation  is  re¬ 
stricted  to  the  semigroup  *U  of  transition  graphs  on  Nn< 

Theorem  2 

There  exist  nonisomorphic  transition  graphs  that  are 
similarly  represented. 

Proof:  Consider  the  following  two  graphs  in  \l 


1 
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and  suppose  that  the  representation  field  is  the  reals.  Then  one  can 
show  that  Tq  and  T^,,  have  the  same  nontrivial  invariant  polynomials 
(invariant  factors),  namely 
3  2 

x  - x  and  x 

and  so  ~  T^,,.  As  G  and  G:  are  obviously  nonisomorphic,  the 
theorem  holds. 

Thus,  even  for  this  relatively  restricted  class  cf  giapns,  a  com¬ 
plete  set  of  similarity  invariants  for  the  representing  linear  transfor¬ 
mations  (e.  g.  their  invariant  polynomials  or  elementary  divisors)  fails 
to  yield  a  complete  set  of  isomorphism  invariants.  The  investigation 
summarized  below  is  concerned  with  the  discovery  of  just  why  this  is 
so,  the  main  result  being  a  graphical  characterization  of  the  structural 
information  conveyed  by  any  complete  set  of  similarity  invariants. 

Summarizing,  first  of  all,  some  well  known  results  regarding 
the  general  structure  of  transition  graphs,  every  (weak)  component  of 
a  transition  graph  is  either  a  flower  (a  weakly  connected  transformation 
graph)  or  a  tree-to-a-point.  (Trees-to-a-point  will  subsequently  be 
referred  to  as  "trees. ")  Also,  transition  graphs  are  obviously  unipathic 
and  consequently  we  can  use  the  notation  [x,  y]  to  denote  a  path  from  x 
to  y.  f  [x,  y]  will  denote  the  length  cf  path  [x,  y].  A  cycle-point  of  a 
transition  graph  is  any  point  that  lies  in  a  cycle.  A  tree-point  is  any 
point  that  is  not  a  cycle-point.  Thus  every  point  cf  a  transition  graph 
G  is  a  cycle-point  iff  G  is  a  permutation  graph;  every  point  of  G  is  a 
tree-point  iff  G  is  a  forest.  The  period  of  a  flower  G  is  the  number  of 
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cycle-points  of  G  (i.  e.  the  length  of  its  unique  cycle).  The  root  of  a 
tree  G  is  the  unique  point  of  G  having  outdegree  0. 

If  G  is  a  transition  graph  and  x  is  a  point  of  G,  let  C(x)  denote 
the  (weak)  component  determined  by  x.  Then  the  notion  of  "height, " 
as  usually  defined  for  trees,  can  be  extended  to  transition  graphs  as 
follows : 

definition  1 

If  G  =  (X,y)  is  a  transition  graph  and  x  e  X  then  the 
height  h(x)  of  x  is  defined  as  follows  : 

i)  If  C(x)  is  a  fiower  then 

h(x)  =  min  {f[x,  y]|y  is  a  cycle-point  of  C(x)} 

ii)  If  C(x)  is  a  tree  then 

n(x)  =  f[x,  Xq]  where  xQ  is  the  root  of  C(x). 

(Note  that  h(x)  =  0  if  and  only  if  x  is  either  a  cycle-point 
or  a  root. ) 

Definition  2 

The  height  h(G)  of  a  transition  graph  G  is  the  maximum 
height  of  any  poini  of  G. 

For  connected*  transition  graphs  (i.  e.  trees  and  flowers),  we 
find  that  the  invariant  polynomials  of  the  representing  transformations 
are  intimately  related  to  the  heights  of  certain  points  in  the  corresponding 

^"Connected"  will  subsequently  mean  ’’weakly  connected"  unless  other¬ 
wise  qualified. 
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graphs.  This  important  relationship  can  be  expressed  in  the  form  of  an 
algorithm  for  computing  the  invariant  polynomials  of  directly  from 
the  structure  of  G.  If  G  =  (X.y)  is  a  transition  graph  let  R(G,  x)  denote 
the  reachable  set  of  x,  i.  e. 

R(G,  x)  =  {y|[x,  y]  is  a  path  of  g} 

and  if  Y  is  a  proper  subset  of  X  let  G-Y  denote  the  removal  of  Y  from 
G,  i.  e. 

G  -  Y  =  G  restricted  to  the  set  of  points  X  -  Y. 

Then  given  any  transition  graph  G  we  define  a  sequence  of  subgraphs 

Gp  Go, . . . ,  Gt 

as  follows : 

i)  G1  =  G 

ii)  If  xA  is  a  point  of  maximum  height  in  Gi  =  (Xj,  y .)  and  RtG^  x.)  f  Xj 
then 

Gi+1  =  Gi  '  R^Gr  XP* 

Otnerwise  the  sequence  terminates,  that  is,  =  G^  We  say 
that  such  a  sequence  is  derived  from  G  and  although  a  derived  sequence 
is  not  necessarily  unique  (even  up  to  isomorphism)  we  obtain  the  follow¬ 
ing  important  result. 
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Theorem  3 

If  G  is  a  connected  transition  graph  and  Gp  G2’  ’  * ' ’ ^t  is  a  se¬ 
quence  of  subgraphs  derived  from  G  then  the  representing  linear 
transformation  has  t  nontrivial  invariant  polynomials 
i=l,  2, . . . ,  t,  which  can  be  graphically  determined  as  follows : 
i)  If  G  is  a  flower  of  period  r  then 

h(Gj)+r  h(Gi) 

^(x)  =  X  -  x 

If  G  is  a  tree  then 

MG^+l 

^1(x)  =  x 
ii)  If  t  >  1  then 

h(G.)+l 

i|/.(x)  =  x  ,  i=2,  3, . . , ,  t. 

The  proof  of  the  theorem  is  based  on  the  classical  decomposition 
of  a  vector  space  V  (relative  to  a  linear  transformation  T  on  V)  into 
cyclic  subspaces  V,,  Vg* . . . ,  Vt  such  that  the  minimum  polynomial  of 
coincides  with  the  i-th  nontrivial  invariant  polynomial  of  T.  The 
process  of  forming  a  derived  sequence  of  subgraphs  parallels  this  de¬ 
composition  process  where  points  of  maximum  height  correspond  to 
vectors  which  generate  the  various  cyclic  subspaces.  The  graph  ob¬ 
tained  on  removing  a  maximum  reachable  set  x.)  corresponds  to 
the  linear  transformation  T  induced  by  T  on  the  quotient  space 
V/Vj  ©  Vg  ©  •  •  •  ®  V-  Therefore,  although  a  somewhat  lengthy  proof 
is  required  to  take  care  of  all  the  details,  the  verification  is  conceptually 
rather  straightforward. 


9 


T 


To  illustrate  the  theorem,  consider  the  following  transition 
graph  which  is  a  flower  of  period  2  on  12  points 


Forming  a  derived  sequence  of  subgraphs  : 

G  |  =  G 

and  as  h(3)  =  h(G^)  (we  could  have  also  chosen  point  7)  and 
RlGp  3)  =  { 1,  2,  3,  4,  5,  6}  we  have : 


2 
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As  h(7)  =  h(G2) : 


Removing  R(Gg,  11), 


and  as  R(G4,  10)  =  { 10} ,  the  process  terminates.  Accordingly,  T^, 
has  four  nontrivial  invariant  polynomials,  namely 

^(x)  =  x6  -  x4 
^2(x)  =  x3 
^3(x)  =  x2 
and  ^4(x)  =  x. 

If  we  remove  the  connectedness  constraint,  the  procedure  of  the 
previous  theorem  cannot,  in  general,  be  applied.  However,  if  we  re¬ 
quire  that  the  transition  graph  be  a  forest*  then  such  a  generalization 
is  possible,  that  is  : 


In  what  follows  we  will  use  the  term  "forest"  to  mean  "transition 
graph  forest. "  Thus  G  is  a  forest  iff  every  component  of  G  is  a 
tree-to-a-point. 


92 


Theorem  4 

If  G  is  a  forest  and  Gg, . . . ,  Gt  is  a  sequence  of 
subgraphs  derived  from  G  then  has  t  nontrivial 
invariant  polynomials  i //.(x)  where 
h(G.)+l 

^(x)  =  x  ,  i=l,  2, . . . ,  t. 

In  order  to  obtain  a  graphical  characterization  of  the  invariant 
polynomials  for  arbitrary  transition  graphs,  we  first  define  two  se¬ 
quences  of  numerical  invariants  that  relate  directly  to  the  structure 
of  a  transition  graph  G.  If  x  is  a  tree -point  of  G  the  depth  d(x)  of  x 
is  the  length  of  the  longest  (directed)  path  to  x,  i.  e. 

d(x)  =  max{  f  [y,  xj  I  [y,  x]  is  a  path  of  G} . 

Accordingly 

Definition  3 

If  G  e  (JL  then  the  depth  sequence  of  G  is  the  sequence 
6  (G)  =  (dg,  dy  . . . ,  dfl_  j) 
where 

d,  =  the  number  of  tree-points  x  of  G  such 

J 

that  d(x)  =  j, 
j—0,  1, . . . ,  n- 1. 

Thus  if  6  (G)  =  (dQ,  dj^ . dn_  jj,  the  sum 

n-1 


j=0 

is  simple  the  total  number  of  tree-points  of  G.  In  particular, 
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if  G  is  a  forest  then  t=n;  if  G  is  a  permutation  graph, 
t=0. 

A  second  sequence  describes  the  cycle  structure  of  a  transition 
graph  and  is  defined  as  follows : 

Definition  4 

If  G  t  ^  the  period  sequence  of  G  is  the  sequence 

ff(G)  =  (rj>  r2>  •  •  •  >  rn) 
where 

Tj  =  the  number  of  components  of  G  that  are 
flowers  of  period  j, 
j=  1,  2, . . . ,  n. 

Note  that  when  G  is  a  permutation  graph,  tt(G)  corresponds 
to  the  usual  description  of  cycle  structure  for  permutations. 
At  the  other  extreme,  if  G  is  a  forest  then  7r(G)  =  (0,  0, . , . ,  0). 

Example :  The  transition  graph 


g: 


6 

n. 


1  2 
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has  depth  sequence 

5(G)  =  (5,  1,  1,  1,-0,  0,  0,  0,  0,  0) 
and  period  sequence 

57(G)  =  (0,  1,  0,  0,  0,  0,  0,  0,  0,  0). 

The  study  of  transition  graphs  decomposes  rather  naturally  into 
the  study  of  two  important  subclasses,  namely  forests  and  permuta¬ 
tion  graphs.  K  we  suppose  first  that  G  is  a  forest,  it  follows  that  T_ 

G 

is  nilpotent  and,  accordingly,  for  some  integer  1  <  t  <  n,  has  t 
invariant  polynomials 


where  e ^  >  ei+1  (1  <  i  <  t).  (The  integers  ei  are  simetimes  re¬ 
ferred  to  as  the  indices  of  a  nilpotent  linear  transformation. )  By 
applying  theorem  4,  the  indices  of  a  representing  nilpotent  transfor¬ 
mation  T  can  be  related  directly  to  the  structure  of  G  as  follows 
G 

Theorem  5 

If  G  is  a  forest  and  T^  has  indices  e^  e^, . . . ,  e{ 
then  the  depth  sequence  of  G  is 

5(G)  =  (do,dr'"’dn-l* 
where  d.  =  I  {i!  e.  >  j}l ,  j=0,  1, . . . ,  n-1. 

In  other  words  the  number  of  points  of  G  having  depth 

j  is  equal  to  the  number  of  invariant  polynomials  of  T„ 

G 

having  degree  greater  than  j. 

Turning  this  result  around  we  can  show,  conversely,  that  the 
depth  sequence  uniquely  determines  the  invariant  polynomials  of 

G 
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that  is, 

Theorem  6 

If  G  is  a  forest  with  depth  sequence 
5  (G)  =  (dg,  dp  . . . ,  d^_  j) 

then,  for  each  integer  i  (1  <  i  <  n),  i  occurs  exactly 

mi  =  di-l  '  di 
times  as  an  index  of  Tq. 

Combining  the  previous  two  results  we  have  proved  that  forests 
are  similarly  represented  if  and  only  if  they  have  the  same  depth 
sequence,  i.  e. 

Theorem  7 

If  G  and  G'  are  forests  then  TQ  ~  T^,  if  and  only  if 
5(G)  =  5(G')- 


This,  then,  completes  the  graphical  characterization  of  similarity 
for  forests. 

To  this  point,  the  transition  graphs  considered  have  been  such 
that  similarity  invariants  of  their  corresponding  linear  transformations 
do  not  depend  on  the  nature  of  the  representation  field.  However,  when 
we  consider  transition  graphs  having  nontrivial  period  sequences  (i.  e. , 
multi- flower  transition  graphs)  this  is  no  longer  the  case.  To  illus¬ 
trate  this  fact,  consider  the  following  two-component  permutation 
graph : 


0 


1 


2 
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K  the  representation  field  F  is  a  field  of  characteristic  0  (say,  the 
reals)  then  the  characteristic  polynomial  <£_  is  given  by: 

4>_  (x)  =  x2  -  2x  +  1. 

G 

On  the  other  hand,  if  F  =  Fg,  the  2 -element  field,  then 


2  . 

$  fx)  =  x  +  1. 
G 


Consequently,  if  G  is  compared  with  the  connected  permutation  graph 


which,  over  any  field,  is  represented  by  a  transformation  having 
the  characteristic  polynomial 

<f>  (x)  =  x2-  1, 

G' 

if  F  has  characteristic  0  then  we  see  that  the  nonisomorphic  graphs 
G  and  G'  can  be  distinguished  by  their  corresponding  characteristic 
polynomials.  On  the  other  hand,  if  the  representation  field  is  Fg, 
we  observe  that  and  T^,  have  the  same  characteristic  polynomial 
(i.  e.  G  and  G'  are  cospectral  over  Fg).  An  important  questions,  there¬ 
fore,  is  whether  the  graphical  interpretation  of  a  complete  set  of  sim¬ 
ilarity  invarlatns  (the  characteristic  polynomial  is  generally  incom¬ 
plete)  will  likewise  depend  on  the  choice  of  representation  field.  We 
begin  the  investigation  of  this  question  by  graphically  formulating  the 

elementary  divisors  of  T~  relative  to  the  various  possible  choices  of 

Cj 


■»  K;tf<f 
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a  representation  field  F. 

In  determining  the  elementary  divisors  of  Tq  we  note  first  of 
all  that  we  can  restrict  our  attention  to  prime  fields  (a  field  is  prime 
if  it  contains  no  proper  subfields).  This  is  possible  since  is  de¬ 
fine  d  in  terms  of  the  scalars  0  and  1  and,  consequently,  if  F  is  a  re¬ 
presentation  field  of  characteristic  k  (k=0  or  some  prime  p)  is 
also  over  the  prime  subfield  Fk  of  F.  As  two  linear  transformations 
are  similar  over  Fk  if  and  only  if  they  are  similar  over  any  exten¬ 
sion  of  Fk  (i.  e. ,  any  field  of  characteristic  k),  no  loss  of  generality 
will  result  from  such  a  restriction. 

Let  Fk  denote  a  prime  field  of  characteristic  k  and  let  $.(x)  de¬ 
note  the  i-th  cyclotomic  polynomial  (over  Ffc)  where  i  is  any  positive 
integer  not  divisible  by  k  (k  i  i).  (If  k=0,  <^(x)  is  defined  for  all 
i  >_  1.  ^(x),  by  definition,  is  the  polynomial  whose  roots  are  all  the 

primitive  i-th  roots  of  unity  found  in  any  extension  of  Fk>)  If  we 
suppose  now  that  G  is  a  permutation  graph  and  consider  first  the  case 
where  F  =  Fq  =  Q  (the  rational  numbers)  the  elementary  divisors  of 
Tq  can  be  graphically  determined  as  follows: 

Theorem  8 

If  G  is  a  permutation  graph  with  period  sequence 
jr(G)  =  (rr  r2 . r^) 

and  the  natural  representation  is  over  Q  then,  for  all 
i  such  that  I  <  i  <  n,  the  cyclotomic  polynomial  4>^(k) 
occurs  exactly 
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m‘  *  .ft  ri 


times  as  an  elementary  divisor  of  T-,.  Moreover,  when 
taken  in  their  totality,  these  are  all  the  elementary  di¬ 
visors  of  1’  . 

u 

To  express  a  similar  result  for  fields  of  prime  characteristic  p, 
if  j  is  some  positive  integer,  let  e(j)  denote  the  exponent  of  p  the  prime 
decomposition  of  j  (if  p  /  j,  c(j)  =  0).  Thus  j  can  be  written: 

j  =  rP<0> 

where  p  (  j\  If  further  we  let  6  .  .  denote  the  familiar  Kronecker 

J 

delta  then 

Theorem  9 

If  G  is  a  permutation  graph  with  period  sequence 
ff(G)  =  (rr  r2,  -..rn) 

and  the  natural  representation  is  over  (p  a  prime) 
then,  for  all  i  such  that  1  <  i  <  n  and  p  /  i,  each  pri¬ 
mary1  factor  of  the  polynomial 
k 

*i(x)P 

occurs  exactly 


m(k)  v  . 

m .  =  0 ,  ,..r, 

1  ifj  k,e(j)  j 

times  as  an  elementary  divisor  of  TQ  (k=0,  1,  • . .  ,[log„p)). 


In  the  prime  decomposition  of  a  polynomial,  the  primary  factor  are 
the  various  powers  of  the  prime  factors. 
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Moreover,  v-hen  taken  in  their  totality,  these  are  all  the 

elementary  divisors  of  T„. 

G 

For  each  of  the  formulations  given  by  the  theorems  8  and  9,  if 

(k) 

one  now  supposes  that  the  inegers  m.  or  m)  '  are  given  and  regards 
the  formulae  as  a  system  of  equations  in  the  n  unknowns 

rl’  r2’ ‘"’V 

it  can  be  shown,  in  each  case,  that  the  system  has  a  unique  solution. 
(The  proof  for  the  characteristic  0  case  is  immediate  whereas  the 
proof  for  the  characteristic  p  case  is  somewhat  more  involved. )  This 
proves 

Theorem  10 

If  G  is  a  permutation  graph  then  over  an  arbitrarily  chosen 
prime  field  (and  hence  over  any  field)  the  elementary  di¬ 
visors  of  Tq  uniquely  determine  the  period  sequence  of  G. 
Combining  the  previous  three  results,  it  follows  that  two  per¬ 
mutation  graphs  are  similarly  represented  if  and  only  if  they  have  the 
same  period  sequence,  i.  e. , 

Theorem  It 

If  G  and  G'  are  permutation  graphs  then  T~  ~  T„, 

G  G 

if  and  only  if  7?(G)  =  tj (G ')- 

Having  graphically  characterized  similar  representations  'or 
forests  in  theorem  7,  and  permutation  graphs  m  theorem  11,  we  now 
obtain  a  general  solution  by  showing  that  a.i  arbitrary  transition  graph 
can  always  be  analyzed  in  terms  of  a  suitably  determined  forest  and/or 
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permutation  graph.  Let  G  =  (X,y)  be  a  transition  graph  and  let  X-, 

r 

denote  the  set  of  all  tree-points  of  G  and  Xp  the  set  of  all  cycle- 
points  of  G.  Then 
Definition  5 

The  reduction  G  of  a  transition  graph  G  =  (X,  y)  is 
the  graph 

G  =  (X,  y) 
vinere 

r  =  y  -  {(x,  y)  I  (x,  y)  e  y,  x  e  Xp,  ye  X^. 

In  short,  G  is  the  result  of  removing  all  the  lines  of  G  that  are 
from  a  tree-point  and  to  a  cycle-point.  Alternatively,  if  we  iet  G„ 

r 

denote  the  restriction  of  G  to  X_  (assuming  X.,,  £  0)  and  let  G~  de- 

r  r  xr 

note  the  restriction  of  G  to  Xp  (assuming  Xp  ^  0)  then  GF  is  a  forest, 
Gp  is  a  permutation  graph  and  G  can  be  described  as  fol’ows: 

f  G„  =  G  if  G  is  a  forest 

I  F 

G  =  <  Gr)  =  GifGisa  permutation  graph 

G„  +  Gn  otherwise. 

^  r  Jr 

Thus  any  transition  graph  that  contains  both  tree  and  cycle  points  re¬ 
duces  to  the  (direct)  sum  of  a  forest  and  a  permutation  graph.  The 
justification  of  this  reduction  is  complete  once  we  establish  the  follow¬ 
ing  important  property. 

Theorem  12 

If  G  is  a  transition  graph  and  G  is  the  reduction  of  G 
then,  over  an  arbitrarily  chosen  representation  field, 
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Given  the  above  property  along  with  the  fact  that 

i)  G  =  G„  +  Gr,  (assuming  G  is  neither  a  forest  nor  a  permutation 

F  P 

graph) 

ii)  The  depth  sequences  6  (G)  and  6  (f  ,)  are  essentially  the  same; 
and 

iii)  The  period  sequences  v(G)  and  ff(Gp)  are  essentially  the  same, 
we  are  able  to  combine  the  characterizations  obtained  for  forests  and 
permutation  graphs  and  establish  the  main  result  of  our  investigation. 
Theorem  13 

If  G  and  G’  are  transition  \raphs  then  T„  ~  T^.  if  and 

u  u 

only  if  6  (G)  =  6  (G')  and  7r(G'  •  ff(G'). 

In  other  words,  two  transition  graphs  with  n  points  are  similarly 
represented  (over  an  arbitrary  field  F)  if  and  only  if  they  agree  both 
in  the  number  points  that  are  tree-points  of  depth  j  (0  <  j  <  n-1) 
and  in  the  number  of  components  that  are  flowers  of  period  k  (1  <  k  <  n). 
Example :  Each  of  the  following  nonisomorphic  transition  graphs  has 
depth  sequence  (2,  1,  0,  0)  and  period  sequence  (1,  0,  0,  0)  and  consequently 
the  representing  transformations  of  all  six  graphs  are  similar. 

1 

*o 

3 
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Applying  the  previous  theorem,  it  becomes  relatively  easy  to 
distinguish  subclasses  of  transition  graphs  for  which  a  complete  set 
of  similarity  invariants  yields  a  complete  set  of  isomorphism  invari¬ 
ants.  To  be  more  precise,  let 

^n  =  {fr  f2’  • '  •  ’ 

denote  the  set  of  isomorphism  invariants  defined  on  as  follows : 
fj(G)  =  i/a(x),  the  i-th  invariant  polynomial  of  T^, 
i=l,  2, . . . ,  n. 

We  refer  to  as  the  set  of  rational  isomorphism  Invariants  and, 

as  a  consequence  of  the  previous  theorem,  obtain  the  following  cri¬ 
terion  for  completeness : 
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Theorem  14 


If  Jb  'll  n  then  ^  is  a  complete  set  of  iso¬ 
morphism  invariants  for  i)  if  and  only  if 
6  (G)  =6  (G’)  and  w(G)  =  tt(G’)  =>  G  =  G’ 
for  all  G,  G'  in  *3  , 

Applying  this  criterion,  it  is  immediate  that: 

Theorem  15 

The  rational  isomorphism  invariants  are  complete 
for  the  class 

5n  =  {GlGe  Un,  h(G)  =  0} 
of  all  zero-height  transition  graphs. 

As  includes  class  of  permutation  graphs  on  Nn  we  have 

Corollary  15.  1 

If  G,  G'  e  J-  then  G  =  G'  if  and  only  if  T_  ~  T_( 

n  u  u 

(over  an  arbitrary  representation  field). 

(The  corollary  i.  known  for  representations  over  a  field  of  character¬ 
istic  0.  On  the  other  hand,  the  generalization  to  fields  of  prime  char¬ 
acteristic  appears  to  be  new.) 

Completeness  classes  containing  giaphs  with  nontrivial  tree  struc¬ 
tures  have  also  been  discovered,  the  most  interesting  being  a  generali¬ 
zation  of  '’homogeneous"  trees.  We  say  that  a  transition  graph  is 
homogeneous  if  points  of  the  same  height  have  ihe  same  indegree  and 
the  same  outdegree.  Every  homogeneous  transition  graph  is  either  a 
forest  or  a  transformation  graph  and  by  a  rather  straightfoiward 
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argument  one  can  prove : 

Theorem  16 

The  rational  isomorphism  invariants  are  complete  for 
the  class  ^  of  homogeneous  transition  graphs. 

Stating  this  result  in  terms  of  the  representing  linear  transformations 
we  have : 

Corollary  16.  1 

If  G,  G'  €  &  then  G  =  G*  if  and  only  if 


Section  6 


Iterative  Network  Realisation  of  Sequential  Machines 
This  research  is  a  continuation  of  the  study  of  a  class  of  itera¬ 
tive  networks  which  was  described  in  the  1967  annual  report.  It  is 
primarily  concerned  with  the  realization  of  sequential  machines  using 
a  special  type  of  cellular  network.  This  section  will  summarize  the 
results  obtained  during  the  past  year.  Proofs  and  additional  details 
can  be  found  in  SvlL  technical  report  no.  06920-22 -T. 

In  the  1967  report,  some  of  the  cellular  networks  that  have  ap¬ 
peared  in  the  literature  were  examined  and  classified  according  to  in¬ 
terconnection  structure  and  cell  type.  The  special  class  of  "iterative 
networks"  was  then  informally  described  in  terms  of  these  classifica¬ 
tions.  This  class  of  networks  was  formalized  by  means  of  the  following 
mathematical  model. 

Definition  1 

An  (abstract)  iterative  network  is  a  6-tuple 

N  (G,  X,  Tj,  S,  Z,  6  )  where: 

X.  G  is  a  finite  group  called  the  interconnection  group 
of  N  with  n  =  I  G  i  ^ . 

2.  X  is  a  subset  of  G  with  k  =  i X 1 . 

3.  tj  is  a  one-to-one  function,  called  the  group  ordering 

for  G,  mapping  Ip  onto  G  such  that  the  image  of  ^  is  X^ 

'  Id  denotes  the  cardinality  of  the  set  G. 
t  T 

^  denotes  the  set  {l.  2, . . . ,  n}. 
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4.  S  is  a  finite  set  of  internal  symbols  with  I  si  >2. 

5.  S  is  a  finite  set  of  inputs  symbols  with  I S I  >1. 

6.  6  is  a  function  mapping  S^xS  into  S  and  called 
the  cell  function  of  N. 

The  following  definition  allows  an  iterative  network  to  be  interpreted 
as  a  sequential  machine. 

Definition  2 

Let  N  =  (G,  X,tj,S,  2,6 )  be  an  iterative  network.  Then 
the  machine  realized  by  N  is  the  sequential  machine 
M(N)  =  (SG,  2,6)  where: 

1.  SG,  the  set  of  all  functions  from  G  to  S,  is  the 
set  of  all  states. 

2.  S  is  the  set  of  input  symbols. 

Q 

3.  6  is  the  state  transition  function  mapping  S  x  2 

r  G 

into  S  ,  and  defined  as  follows.  For  f  in  S  and 

a  in  2  ,  the  successor  state  of  f  under  input  a  is 

the  function  (f.  o) 6  which  takes  an  element  g  of 

G  into  the  element 

[(g*j)f,  •*>  (gxk)f.a]6 

of  S  where  i rj  =  x.  for  i=l,  2, . . . ,  k. 

The  function  6  is  called  the  behavior  of  the  network  N. 
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There  are  several  notations  in  the  literature  of  one  machine 
imitating  another.  When  the  machines  in  question  do  not  have  out¬ 
puts,  then  the  only  thing  which  must  be  imitated  is  the  state  transition 
function.  This  is  usually  done  by  finding  states  or  sets  of  states 
in  the  imitating  machine  which  behave  in  the  same  way,  with  respect 
to  their  respective  transition  functions,  as  the  states  of  the  machine 
being  imitated.  We  will  use  the  following  notion  of  one  machine 
imitating  another. 

Definition  3 

Let  M  and  M'  be  sequential  machines.  Then  M 
is  said  to  realize  M'  if  there  is  a  submachine 
of  M  isomorphic  to  M\ 

This  definition  is  frequently  weakened  even  more  by  only  requiring 
that  M*  be  a  homomorphic  image  of  a  submachine  of  M.  However, 
for  our  use,  the  stronger  version  is  sufficient. 

It  is  by  means  of  this  concept  of  imitation  that  the  abstract 
theory  of  machines  is  usually  linked  to  the  physical  circuits  used 
to  realize  them.  In  our  case,  the  circuit  has  been  modeled  as  an 
iterative  network  which  realizes  sequential  machines  as  follows. 

Definition  4 

Let  M  be  a  sequential  machine.  N  an  iterative  network, 
and  M(N)  the  machine  realized  by  N.  Then  the  network 
N  realizes  the  machine  M  if  and  only  if  M(N)  realizes  M. 
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Note  that  any  iterative  network  N  will  always  realize  M(N),  th£  ma¬ 
chine  realized  by  N.  In  general,  it  will  also  realize  many  other 
machines  as  well.  Hence  the  machine  realized  by  the  iterative  net¬ 
work  N  is  the  sequential  machine  M(N)  (definition  3)  while  a  ma¬ 
chine  realized  by  N  is  any  machine  isomorphic  to  a  submachine  of 
M(N). 

In  order  to  find  an  iterative  network  which  realizes  a  given 
sequential  machine,  it  is  first  necessary  to  find  a  group  which  can 
be  used  as  the  network's  interconnection  group.  The  following  re¬ 
sult  suggests  that  we  should  use  some  subgroup  of  the  machine's 
automorphism  group. 

Theorem  1 

The  interconnection  group  G  of  the  iterative  network 
N  is  isomorphic  to  a  subgroup  of  the  automorphism 
group  of  M(N). 

The  next  result  states  that,  in  fact,  any  subgroup  may  be  used. 

Theorem  2 

Let  M  =  (Q.  Z,  X)  be  a  sequential  machine  and  G 
be  any  group  of  automorphisms  of  M.  Then 
there  is  an  iterative  network  N  which  uses  G 
as  its  interconnection  group  and  which  realizes  M. 
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There  are  two  cases  in  which  Theorem  2  results  in  '’trivial” 
realizations.  The  first  is  when  the  automorphism  group  of  the 
machine  to  be  realized  is  trivial.  This  results  in  a  network  with 
only  one  cell  which  is  as  complex  as  the  machine.  The  second  case 
arises  when  the  cardinality  of  the  set  of  internal  symbols  is  equal 
to  the  number  of  states  in  Q.  In  this  case,  all  of  the  cells  are  as 
complex  as  the  original  machine  and  nothing  is  gained  by  using  more 
than  one  cell. 

It  has  been  shown  that  the  second  case  can  always  be  avoided 
it  the  number  of  states  in  Q  is  greater  than  2.  However,  both 
cases  can  be  avoided  simultaneously  by  extending  Theorem  2  to  ob¬ 
tain  a  "binary  network". 

An  iterative  network  is  called  binary  if  the  cardinality  of  its 
set  of  internal  symbols  is  two.  That  is,  if  N  =  (G,  X,  77,  S,  2, 5  )  is 
an  iterative  network,  then  N  is  a  binary  network  if  and  only  if 
I S I  =2.  In  this  case,  every  cell  has  a  single  output  terminal  which  can  take 
on  only  two  distinct  values.  We  ask  the  following  question.  Given 
an  arbitrary  sequential  machine,  can  we  find  a  binary  iterative  net¬ 
work  which  realizes  it  ?  The  answer  is  given  by  the  following  theorem. 

Theorem  3 

Given  any  sequential  machine  M  =  (Q.  2,  A),  there 
exists  at  least  one  binary  iterative  network  which 


realizes  it. 
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The  proof  of  this  result  involves  first  embedding  the  machine 
to  be  realized  in  a  larger  machine  which  has  a  nontrivial  automor¬ 
phism  group.  Then  the  techniques  used  in  the  proof  of  Theorem  2 
are  used  to  realize  this  machine.  Since  the  relation  of  ma¬ 
chine  realization  is  transitive,  this  resulting  network  will  realize 
the  original  machine. 

Although  the  proof  of  Theorem  3  is  constructive,  it  can  not 
be  considered  a  practical,  design  technique  due  to  the  large  num¬ 
ber  of  cells  in  the  realizing  network.  This  procedure  can,  however, 
be  applied  successfully  to  certain  special  classes  of  machines  to  get 
more  practical  binary  realizations.  In  particular,  the  class  of 
autonomous  machines  and  the  class  of  total  automata  (i.  e.  strongly 
connected  sequential  machines  whose  automorphism  group  is  as 
large  as  possible)  can  be  realized  with  binary  networks  where  the 
number  of  cells  is  the  same  order  of  magnitude  as  the  number  of 
states  in  the  machine.  In  addition,  total  automata  can  be  realized 
with  binary  networks  where  the  number  of  internal  cell  input  ter¬ 
minals  is  equal  to  the  number  of  external  input  symbols. 


Section  7 


Optimal  Sequencing  of  Jobs  Subject  to  Deadlines 
7. 1  Introduction 

Held  and  Karp  [19]  have  developed  a  general  dynamic  program¬ 
ming  algorithm  for  problems  involving  the  sequencing  of  jobs  subject 
to  deadlines.  There  are  a  number  of  these  problems,  however,  for 
which  a  certain  "consistency  principle"  holds.  This  principle  can 
be  stated  as  follows  : 

Given  a  set  of  jobs  which  are  to  be  processed  subject 
to  deadlines,  there  exists  a  linear  ordering  of  the 
complete  set  of  jobs,  such  that  for  any  subset  of 
these  jobs,  an  optimal  sequence  exists  in  which  the 
ordering  of  the  jobs  completed  on  time  is  consistent 
with  the  linear  ordering. 

This  paper  demonstrates  how  this  consistency  principle  can 
be  exploited  to  obtain  solution  methods  which  are  considerably 
more  efficient  than  that  of  Held  and  Karp.  Problems  considered 
are  (1)  a  single  machine  and  jobs  with  individual  deadlines 


ill 
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(Moore  [32]).  (2)  the  same  problem  with  partial  ordering  restric¬ 
tions  on  the  sequence,  (3)  two  machines  in  series  and  jobs  with 
a  common  deadline  (Johnson  [20  ]),  and  (4)  a  single  machine,  jobs 
with  a  common  deadline,  and  individual  linear  deferral  costs 
(McNaug'aton  [ 30]  and  Smith  [4lj).  These  problems  are  dealt  with 
in  Sections  2,  3,  4,  and  5,  respectively.  Two  concluding  sections 
discuss  additional  problems  with  partial  ordering  restrictions  and 
multi- machine  generalizations. 

7.  2  Single  Machine  Multiple  Deadlines 

Let  there  be  given  jobs  J2, . . . ,  Jn<  For  each  job  J.,  let 
t.  denote  its  processing  time  on  a  single  machine,  D^,  its  deadline, 
and  r  a  regard  which  is  earned  if  processing  of  the  job  is  com- 

x 

pleted  by  the  deadline.  Without  loss  of  generality,  assume  that 
Di  <  Di+1;  for  1=1,2, . . . ,  n-1. 

We  wish  to  find  a  sequence  for  the  jobs  which  maxi¬ 
mizes  the  sv  of  the  earned  rewards.  According  to  Moore  [3z], 
there  exists  an  optimal  sequence  in  which  (1)  the  jobs  completed 
on  time  (meeting  deadlines)  precede  the  tardy  jobs  and  (2)  the  011- 
time  jobs  are  ordered  according  to  their  deadlines,  earliest  dead¬ 
line  first. 

It  follows  that  the  problem  consists  of  making  a  selection  of 
the  jobs  which  are  to  be  completed  on  time.  Given  .ueh  a  selection, 
the  ordering  oi  these  jobs  is  determined  solely  by  eir  deadlines; 
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the  ordering  of  the  remaining  (tardy)  jobs  which  follow  is  arbi¬ 
trary.  The  selection  problem  can  be  formulated  as  an  integer 

linear  program  as  follows : 

n 

Maximize  2  r.x. 

i=l  11 

subject  to  Tx  <  D 
where 

xi  =  1  if  job  J.  is  selected 
=  0  otherwise, 
and  T  and  D  are  of  form 


0 

0 

0 

0 
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As  one  might  expect  from  the  special  structure  of  this  prob¬ 
lem,  it  can  be  solved  by  dynamic  programming  techniques  similar 
to  those  used  for  the  well  known  "knapsack"  problem. 
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Definition:  Let  f  (t)  =  the  maximum  attainable 
total  reward  fox' a  selection  of  jobs  from  the 
set  of  jobs  -fjj  . . .  «L},  subject  to  the  con¬ 
straint  that  the  completion  time  of  no  job  is 
later  than  t. 

A  recursion  relation  for  f.(t)  can  be  formulated  as  follows : 

1)  Consider  t  <  D..  If  there  exists  an  optimal  schedule 
in  which  J.  is  completed  on  time,  then  J.  can  be  the 
last  on  time  job  and  f.(t)  =  f.  -  (t-t.)  +  r..  If  no  such 
schedule  exists,  then  f.(t)  =  L  ^(t). 

2)  For  t  >  D.  >  . . .  >  D1 ,  f.(t)  =  f.(D.)  and  can  be  com- 
puted  in  the  manner  indicated  above. 

Hence,  for  i=l,  2,  . . . ,  n : 
fj(t)  =  fjtDj)  for  t  >  Di 

fj(t)  =  max  {fj.jlt),  rj  +  f  jft-t.)}  for  0  <  t  <  D; 
subject  to  the  boundary  conditions 

fj(0)  =  0 

f0(t)  =  0 

i.(t)  =  -  oo  for  t  <  0 

The  maximum  attainable  total  reward  for  the  complete  set  of 
jobs  is  given  by  fn(Dn). 

Assuming  that  all  of  the  processing  times  and  deadlines  are 
integers,  the  length  of  the  computation  grows  no  more  rapidly  than 
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nDn,  i.  e.  proportional  to  the  product  of  the  number  of  jobs  and 
the  longest  deadline. 

Note  also  that  the  problem  is  symmetric  with  respect  to  time. 
This  means  that  with  slight  modification  of  the  recursion  equations, 
one  can  just  as  well  solve  a  problem  in  which  the  times  denote 
earliest  possible  starting  times  for  the  jobs,  and  one  determines  a 
sequence  which  maximizes  the  sum  of  the  earned  rewards,  subject 
to  a  common  deadline,  t,  for  all  jobs. 

7.  3  Single  Machine  Problems  with  Partial  Ordering  Restrictions 

In  the  single  machine  problem  just  discussed,  it  was  assumed 
that  any  sequence  for  the  jobs  was  feasible.  An  additional  compli¬ 
cation  arises,  however,  when  it  is  required  that  the  sequence  for 
the  jobs  must  be  consistent  with  a  given  partial  ordering  Q,  which 
may  be  due  to  technological  restrictions  of  various  kinds. 

First  we  shall  develop  necessary  and  sufficient  conditions 
for  the  existence  of  a  sequence  in  which  all  jobs  are  completed 
on  time,  consistent  with  the  given  partial  ordering  restrictions. 

These  conditions  generalize  the  result  of  Smith  [  41 1 
that  an  unrestricted  set  of  jobs  can  all  be  completed  on  time  if  and 
only  if  they  are  completed  on  time  when  they  are  sequenced  ac¬ 
cording  to  deadlines. 

Again  let  there  be  given  jobs  J^,  Jg.  ■  •  ■  -  where  job  J. 
has  processing  time  t.  and  deadline  D..  with  an  arbitrary  partial  order¬ 
ing  Q.  The  object  is  to  find  a  linear  ordering  L 
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such  that  all  jobs  can  be  processed  on  time  consistent  with  Q 
iff  they  are  on  time  in  the  sequence  defined  by  L. 

Assume,  without  loss  of  generality,  that  <  Dg  <  .  •  •  <  Dn 
and  for  each  job,  J.,  define  a  vector,  V1,  as  follows  : 

v‘  ■  <vVvl2 . V‘n> 

where  =  1  if  (J.,  Jfc)  e  Q 

=  0  otherwise. 

Let  L  denote  the  relation  on  the  jobs  obtained  by  ordering 
these  vectors  lexicographically  from  the  largest  to  the  smallest. 

Lemma  1 :  L  is  a  linear  ordering. 

Proof :  Since  L  is  determined  by  a  lexicographic  ordering,  it  is 
known  to  be  both  reflexive  and  transitive.  Hence,  it  is  only  neces¬ 
sary  to  show  that  L  is  antisymmetric.  Consider  a  pair  of  jobs, 

J.  and  J.  ,  i^k  where  V1  >  Vk.  We  must  show  that  Vi  >  V^. 

IK  — 

i  v  k  i 

Suppose  V  =  V  ,  then  V  .  =  V  .  =  1,  since  Q  is  a  partial  ordering. 

1  K 

Hence  (J.,  J, )  and  (J.  ,  J.)  €  Q,  a  contradiction  since  Q  i"  a 

1  K  K  1 

partial  ordering  and  i^k. 

Hence  Vi  /  VR  and  ">  Vk- 

Lemma  2  :  L  is  consistent  with  Q. 

Proof:  If  (J.,  J.)  €  Q.  then  Vk  =  1  implies  that  Vi  =  1. 

IK  Q  0 

Further,  Vk  =  0  while  V1.  *  1 .  Hence  V1  >  ^  and  (J.,  J, )  c  L. 
i  l  '  i  k 
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Theorem  :  All  jobs  in  the  set  J  can  be  completed  on  time  con 
sistent  with  the  partial  ordering  Q  iff  they  are  on  time  in  the 
sequence  defined  by  L. 


Proof :  If  -  Obvious 
Only  if : 

Consider  an  arbitrary  sequence  S  =  (J.  . . .  J.  )  in  which  all  jobs 

X 

1  n 

are  completed  on  time  consistent  with  the  partial  ordering  Q, 

where  S  is  distinct  from  L.  Then  there  exists  a  pair  of  adjacent 

jobs,  J.  J.  in  the  schedule  S  where 
he  he+l 

y1k+l  >  yhc 

Thus,  there  must  be  a  first  position,  say  q,  in  which  the  vector 

hc+i  he 

V  contains  a  1,  but  the  vector  V  contains  a  0.  Clearly,  q*  ifc 

and  there  are  two  cases  to  consider: 


1) 


q  =  1 


k+l 


2) 


Then  D.  <  D.  and  a  new  schedule,  S1  may  be 
he+l  “  \ 

defined  from  S  by  interchanging  J.  and  J.  in  which 

he  he+! 

J.  and  J.  are  still  on  time. 
lk  he+l 


q  v  1 


k+r 


Then  job  J  has  D  <  D.  ,  D.  and  (J.  ,  J  )  e  Q. 

q  q  he+l  he  he+l  q 


Hence  J  follows  both  J. 
q  1 


and  J.  in  the  schedule  S  and  a 
k+l  he 


new  schedule,  S.  may  be  defined  from  S  by  interchanging  J.  and 

lk 

J.  in  which  J.  and  J, 
he+l  he  he+l 


are  still  on  time. 
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If  Sj  is  L,  we  are  done.  Otherwise  the  above  process  is 
repeated  to  obtain  schedules  S«  . . .  S  where  S  is  L. 

2  p  p 

Q.E.D. 

This  result  can  be  used  to  obtain  a  dynamic  programming 
solution  to  the  following  problem.  All  jobs  are  to  be  completed 
on  time,  subject  to  the  partial  ordering  restriction  Q,  where  for 
each  job  J.  there  is  a  choice  of  one  of  m  processing  times  t^, 
t^.,  . . . ,  tm.  with  associated  rewards  r*.,  r^., . . . ,  r111^  (It  is 
reasonable  to  assume  that  the  rewards  would  be  proportional  to 
the  processing  times. ) 

Since  all  jobs  are  to  be  completed  on  time  consistent  with  Q, 

the  only  sequence  that  need  be  considered  is  that  defined  by  L. 

Consequently,  let  us  now  assume  that  the  jobs  are  reindexed  accord 

ing  to  L,  and  let  f.(t)  denote  the  maximum  attainable  total  reward 

associated  with  the  first  i  jobs  in  the  sequence  L,  subject  to  the 

condition  that  no  job  is  completed  !ater  than  time  t. 

A  recursion  equation  for  f.(t)  is  as  follows.  For  i-1, 2,  ....  n 

f  (t)  =  f.(D.)  for  t  ">  D. 
r  '  r  l  i 

=  max  { rJ.  -t-  f.  ,(t-tJ.)j  fur  0  \  t  .  D., 

j=l .  2 . rn 

subject  to  the  boundary  conditions 
f.(0)  =  0 


fj(t)  -  -  x  for  t  0  . 
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The  case  in  which  it  is  not  required  that  all  jobs  are  com¬ 
pleted  on  time  is  discussed  in  Section  6. 


7.  4  Two  Machines  in  Series,  Common  Deadline 

We  now  deal  with  the  problem  of  two  machines  in  series 
and  a  common  deadline  where  each  job  must  be  processed  by  both 
machines,  in  sequence.  Consider  a  set  of  jobs,  J  =  {j  ,  . . . , J  ] 
where  for  each  job  J.,  a^  denotes  its  processing  time  on  the  first 
of  two  machines,  and  b.  its  processing  time  on  the  second.  Let 
r.  denote  a  reward  which  is  earned  if  job  <L  is  completed  by  the 
second  machine  by  a  deadline  T  (common  to  all  jobs).  The  prob¬ 
lem  is  to  find  a  sequence  for  the  set  of  jobs  which  maximizes  the 
sum  of  the  earned  rewards. 

It  follows  from  the  results  of  the  previous  chapter  and  those 
of  Johnson  f 20 !  that  there  exists  an  optimal  sequence  in  which  (1) 
the  jobs  completed  on  time  precede  the  tardy  jobs:  (2)  the  jobs 
are  processed  in  the  same  order  by  both  machines :  and  (3)  the 
on-time  jobs  are  ordered  according  to  the  following  relation  : 

Job  J  precedes  J  only  if  min  !a  ,  b  !  .  min  {a  ,b  !. 

p  r  q  '  !  p  q1  _  •  q  p 

Once  again,  the  problem  consists  of  making  a  selection  of 
the  jobs  which  are  to  be  completed  on  time.  Given  such  a  selection. 
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the  ordering  of  these  jobs  is  determined  by  Johnson's  relation; 
the  ordering  of  the  remaining  jobs  which  follow  is  arbitrary. 
Without  loss  of  generality,  assume  that 

min{a.,b.  * }  <  min  {a.  ..b.},  for  j=l, . . . , n-1. 

1  j  j+l  —  j+i  j 

Let  f.(tj,  tg)  =  the  maximum  attainable  reward  for  a  selec¬ 
tion  of  jobs  from  among  Jj , . . . ,  J  subject  to  the  constraint  that 
the  completion  time  of  no  job  is  later  than  time  t^  on  the  first  ma¬ 
chine  or  tg  on  the  second.  Following  the  type  of  argument  used  in 
Section  2,  a  recursion  relation  for  f ,  tg)  can  be  formulated  as 
follows . 

For  i=0, 1,  2,  . . . ,  n : 

f.(tr  t2)  =  max  (fi_1(t1,  t2),  r.  +  f.^minj^-a.,  tg-a.-b.},  tg-b.)} 
subject  to  the  boundary  conditions  : 

VW  =  0 

L(0,  0)  =  0 

fj(ti,t2)  =  -oo  (tj  <0  or  t2  <  0) 

The  maximum  attainable  total  reward  for  the  complete  set  of 

jobs  is,  of  course,  given  by  fn(T,T).  Assuming  all  processing 

times  are  integers,  the  length  of  the  computation  grows  no  more 
2 

rapidly  than  nT  . 
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7.  5  Single  Machine,  Common  Deadline,  Linear  Deferral  Costs 

As  a  further  example  of  the  consistency  principle,  consider 
the  problem  of  a  single  machine,  a  common  deadline,  and  linear 
deferral  costs. 

Consider  a  set  of  jobs  J  =  {j^, . . .  ,  Jn),  where  for  each  job 
JL,  a^  denotes  its  processing  time  and  r.  a  reward  which  is  earned 
only  if  the  job  is  completed  by  a  deadline  T  (common  to  ail  jobs). 

In  addition,  let  pi  denote  a  linear  deferral  cost  coefficient.  If  the 
job  J.  is  completed  at  a  time  t  <  T,  the  net  reward  earned  for  that 
job  (reward  minus  deferral  cost)  is  r^-p.t. 

Consider  the  position  of  a  contractor  who  is  free  to  accept 
or  reject  jobs.  For  each  job  «L  which  is  accepted  and  completed 
prior  to  the  deadline,  a  reward  r.  is  earned.  However,  the  deferral 
of  the  job  causes  a  cost  to  be  incurred  which  is  determined  by  the 
coefficient  p. .  What  selection  of  jobs  maximizes  profit? 

Given  any  selection  of  jobs,  such  that  the  sum  of  their 


processing  times  is  no  greater  than  T.  the  jobs  should  be  ordered 
according  to  the  ratios  p./a;,  the  job  with  the  largest  ratio  being 
processed  first.  This  result  has  been  found  by  McNaughton  [30 ] 


and  Smith  [41 


pi  pi+l 

Without  loss  of  generality,  assume  —  '  - 

a .  —  a.  . 

l  l+l 


.  Let 


f.(t)  =  the  maximum  attainable  total  net  profit  for  a  selection  of 
jobs  from  among  J^,  Jg,  . . . .  J^,  subject  to  the  constraint  that  the 


starting  time  of  the  first  job  is  t  and  the  last  job  is  completed  no 
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later  than  T. 

An  appropriate  set  of  Recursion  equations  is  as  follows  for 

t  >  0. 

f.(t)  =.  max  {fi_1(t)r.  -  p^t+a.)  +  ^(t+a.)} 

subject  to  the  boundary  conditions 

f.(T)  =  0 

f0(t)  -  o 

f.(t)  =  -  oo  for  t  >  T. 

l 

The  maximum  attainable  total  net  profit  for  the  complete  set  of  jobs 
is  given  by  fn(0).  Assuming  all  processing  times  are  integers,  the 
length  of  the  computation  grows  as  nT. 

There  is  an  interesting  variation  of  this  problem  in  which  the 
deadline  is  not  controlling.  E.  g. ,  T  is  as  large  as  the  sum  of  all 
the  processing  times.  In  this  case,  the  selection  of  jobs  is  con¬ 
trolled  solely  by  the  question  of  whether  or  not  jobs  can  be  com¬ 
pleted  before  their  deferral  costs  exceed  their  rewards. 

7  6  Additional  Partial  Ordering  Considerations 

It  would  be  desirable  to  be  able  to  extend  the  results  of  Sec¬ 
tions  2.  4,  and  5  to  the  situation  in  which  optimal  sequences  are 
to  be  derived  subject  to  the  restriction  that  either  (a)  ail  jobs  are 
ordered  consistently  with  an  arbitrary  partial  ordering  Q  or  (b)  all 
on  time  jobs  are  so  ordered.  (Section  3  presented  such  results 
for  the  case  in  which  all  jobs  were  required  to  be  on  time. ) 
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Unfortunately,  the  results  obtained  above  do  not  generalize  except 
for  very  special  partial  orderings  for  which  the  consistency  prin¬ 
ciple  can  be  shown  to  hold. 

One  such  case  arises  when  the  partial  ordering  induces  a  par¬ 
titioning  of  the  jobs  into  a  sequence  of  equivalence  classes, 

(Ej, . . . ,  Ep)  where 

1)  For  each  equivalence  class,  E.,  Jr,  Jg  e  E^, 

implies  that  J  and  J  are  not  related  by  Q. 

1*  s 

2)  Jf  e  E.,  Jg  e  E ,+1  implies  that  (Jr,  Jg)  e  Q. 

In  this  case,  the  dynamic  program  ning  algorithms  previously 
defined  may  be  applied  to  each  equivalence  class  individually  and 
the  optimal  sequence  can  be  obtained  by  concatenating  the  recusing 
sequences  for  Et .  E«, . . . ,  E  .  (The  individual  solutions,  of  course, 
take  into  account  that  the  starting  time  for  the  first  job  in  the  i-th 
equivalence  class  equals  the  sum  of  the  processing  times  for  the  jobs 
in  the  previous  i-1  equivalence  classes.) 

There  are  also  other  situations  in  which  Q  may  not  be  or  this 
special  form,  but  the  parameters  of  the  problem  are  such  that  the 
consistency  principle  can  be  shown  to  hold. 

7.7  Mac  lane  s  in  Parallel  * 

Each  of  the  problem  formulations  and  solution  methods  given 
above  can  be  extended  in  a  very  natural  way  to  the  situation  in 
which  there  are  many  machines,  or  sets  <>i  machines.  ;n  parallel. 
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and  any  given  job  can  be  processed  by  any  given  machine.  In  e'»ch 
such  extension,  the  jobs  that  are  assigned  to  any  given  machine 
are  processed  in  an  order  which  is  consistent  with  the  ordering 
obtained  by  solving  the  associated  single  machine  problem. 

Consider  the  extension  of  the  problem  of  multiple  deadlines 
(Section  2  above).  Let  there  be  given  a  set  of  jobs,  J  =  {j^  . . .  J^}. 
For  each  job  J  ,  let  a.  ,  denote  its  processing  time  on  the  k-th 
of  M  machines  and  r.  ,  a  reward  which  is  earned  if  processing  of 

1,  K 

the  job  is  performed  on  machine  k  and  completed  prior  to  the  dead¬ 
line  for  the  job,  D^. 

As  before,  we  assume  without  loss  of  generality,  that  < 
Di+j,  for  t=l,  2, . . . ,  n-i.  (Note  that  it  is  feasible  to  have  different 
deadlines  on  different  machines.  However,  it  must  be  the  case 
that  D.  <  D.  .  ,  for  all  k. )  Let  f.(t,  ...  t  )  =  the  maximum 
attainable  reward  for  a  selection  of  jobs  from  among  J^,  Jg,  . . . ,  «L, 
subject  to  the  constraint  that  the  completion  time  of  no  job  is  later 
than  tj  for  machine  k.  Now,  we  have,  for  i-0, 1, . . . ,  n : 


t  )  =  f.(t,,t0, 
nr  r  1  2 


-rDi’  ‘k+l  ’ 


,>• if  *k 


-  max  {L_j(tj,  tg,  •  •  •  > 

. Vai,  k . V1 


otherwise 


subject  to  the  boundary  conditions 
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f0(tr  t2, . . . ,  tm)  -  0 

f.(0,0,...,0)  =  0 

f.(tj,t2, .  ..,tm)  =  -co  if  any  ^  <  0 

The  length  of  the  computation  implied  by  these  recursion 

equations  grows  as  mnD™  •  Some  saving,  of  course,  can  be 

realized  through  exploitation  of  symmetry  in  the  case  in  which 

all  machines  are  identical,  i.  e.  a.  =  a.  and  r.  =  r.  for 

J,P  J,q  J,P  q 

all  j,  p,  q. 

The  formulation  of  recursion  equations  for  the  extensions 
of  the  problem  described  in  Sections  4  and  5  is  quite  similar, 

2  in  m 

and  results  in  computations  which  grow  as  mnT  m  and  mnT  \ 
respectively.  One  should  compare  the  extension  of  the  deferral 
cost  case  with  the  solution  method  given  by  Rothkopf  [40  ]  for  the 
multi- machine  deferral  cost  problem  without  deadlines.  In  that 
case,  a  computational  growth  of  mnT  1  is  possible. 

We  note  that  in  the  generalizations  of  the  problems  of  Sec¬ 
tions  4  and  5,  it  is  possible  to  have  some  variation  in  the 
characteristics  of  the  individual  machines,  provided  the  existence 
of  a  single  linear  ordering  is  not  interfered  with.  Ln  the  case  of 
sets  of  two  machines  in  series,  it  must  be  possible  to  find  a  linear 
ordering  such  that,  for  all  k, 

min(\  V  V  k>  <  min(ai*l,k-bj,k) 
and  in  the  case  of  linear  deferral  costs, 
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Pj>  k  <  pj+l;k 
aj,k  “  aj+l,k 

The  rewards,  r.  , ,  need  be  related  in  no  particular  way. 
3> K 


Section  8 


THE  THEORY  OF  FORMAL  LANGUAGES  AND  ITS  IMPACT  ON  THE 
DESIGN  AND  IMPLEMENTATION  OF  PROGRAMMING  LANGUAGES 

Leonard  Y.  Liu 


8. 1  Introduction 

Recently,  research  in  the  area  of  formal  languages  has  been 
intensified  because  of  interesting  interpreations  with  respect  to  pro¬ 
gramming  languages.  A  formal  language  is  any  set  of  finite  length 
strings  of  symbols  over  a  finite  alphabet.  The  theory  of  formal 
languages  is  concerned  with  the  description  of  languages,  their  prop¬ 
erties,  structures,  relationships  and  their  recognition.  Some  of  the 
results  of  this  theory  are  of  direct  interest  to  the  designer  of  compu¬ 
ters,  programming  languages  and  compilers.  Some  of  these  results 
are  briefly  reviewed  in  this  section. 

8.  2  Finite  Descriptions  of  Languages 

Formal  languages  were  first  considered  by  mathematical  lin¬ 
guists  as  mathematical  models  for  natural  languages  [  5] .  If  a  natural 
language  had  only  a  finite  number  of  sentences,  it  could  be  completely 
specified  by  a  finite  list.  However,  this  is  not  the  case  .  Almost 
every  meaningful  language  has  an  infinite  number  of  sentences. 
Chomsky  defined  four  types  of  grammars  [3,  4]. 

A  type  0  grammar  is  a  4-tuple  G  =  (N,  E,  P,  S).  N  and  2  are 
two  disjoint  nonempty  sets  of  nonterminal  symbols  and  terminal 
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symlKJls  respectively.  S  is  a  special  nonterminal  symbol  (sentence 
symbol).  P  is  a  finite  nonempty  set  of  production  rules  of  the  form 
a  -  /3.  a  is  any  string  of  nonterminal  symbols  and  terminal  symbols 
with  at  least  one  nonterminal  symbol  in  it.  0  is  any  string  of  termi¬ 
nal  and  nonterminal  symbols.  Let  a,  be  strin£s  of 

terminal  symbols  and  nonterminal  symbols.  If 

u>2  =  2>  and  a  “  P  is  in  P,  then  =>  uv,  is  used  to  denote 

* 

this  fact  (u).,  ===>  w0  is  used  whenever  G  is  understood).  wn  ===>  lo 

if  there  exist  Wq,  Wj, . . . ,  u>n  such  that  =p=>  u>i+^  for  1  <  i  <  n 

* 

(u>q  =>  cun  is  used  whenever  G  is  understood).  The  language  gen¬ 
erated  by  G  is  denoted  by  L(G). 

L(G)  =  {xls  =^=>  x  and  x  is  a  string  of  terminal  symbols}. 
A  type  1  grammar  (context  sensitive  grammar  ),  G,  is  a  re¬ 
stricted  type  0  grammar  in  that  the  length  of  /3  is  longer  than  or  e- 
qual  to  the  length  of  a  for  every  production  rule  a  -  0  in  G.  A  lan¬ 
guage  is  a  context-sensitive  language  if  it  can  be  generated  by  a  con¬ 
text  sensitive  grammar. 

A  type  2  grammar  (context  free  grammar),  J,  is  a  restricted 
type  1  grammar  in  that  every  production  rule  m  G  is  of  the  form 
A  -  0,  where  A  is  a  single  nonterminal  svmbol  and  0  is  a  string  of 
terminal  and  nonterminal  symbols.  A  language  is  called  a  context- 
free  language  if  it  can  be  generated  by  a  context  free  grammar. 

A  type  3  grammar,  G,  £  a  restricted  type  2  grammar  in  that 
every  production  rule  is  o'  the  form  A  -  aB  or  A  *•*  a  where  A,  B  are 
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nonterminal  symbols  and  a  is  a  terminal  symbol. 

It  is  well  known  that  the  context  free  grammars  form  the  back¬ 
bone  of  the  syntax  of  programming  languages.  The  most  represen¬ 
tative  example  of  this  is  the  use  of  ccntext  free  grammar  in  specify¬ 
ing  most  of  the  syntactic  rules  of  ALGOL  60  [34].  Clearly,  this  per¬ 
mits  a  more  precise  and  efficient  specification  of  the  programming 
languages.  However,  attention  should  be  paid  to  the  fact  that  context 
free  grammars  are  not  quite  powerful  enough  to  completely  specify 
most  programming  languages.  Parts  which  are  not  appropriate  for 
context  free  specifications  are  specified  by  English  in  ALGOL  60  [34]. 
On  the  other  hand,  the  context  sensitive  grammars  can  completely 
specify  the  programming  languages.  An  interesting  problem  is  to 
define  some  class  of  languages,  lying  between  the  class  of  context 
sensitive  languages  and  the  class  of  context  free  languages,  that  is 
powerful  enough  to  model  the  programming  languages  and  may  be 
parased  efficiently.  Little  is  known  about  classes  of  languages  which 
lie  between  the  class  of  context  sensitive  languages  and  the  class  of 
context  free  languages.  Indexed  grammars  [l],  programmed  gram¬ 
mars  [39],  and  finite-reversal  pushdown  automata  [27]  specify  classes 
of  such  languages. 

8.  3  Ambiguity 

When  a  programming  language  is  being  designed,  it  is  always 
essential  to  know  whether  the  "interpretation"  of  every  sentence  in 
the  language  is  unique  or  not.  This  idea  has  been  formalized  in  the 
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theory  of  formal  languages.  A  grammar  is  ambiguous  if  there  exist 
two  "distinct"  derivations  of  a  sentence*.  A  language  is  inherently 
ambiguous  if  every  grammar  generating  the  language  is  ambiguous. 

It  has  been  shown  that,  1)  there  does  not  exist  an  algorithm  to  decide 
whether  a  context  free  grammar  is  ambiguous;  and  2)  there  does  not 
exist  an  algorithm  to  decide  whether  a  context  free  language  is  inher¬ 
ently  ambiguous  [2, 12, 13].  These  negative  results  are  important 
since  they  prevent  research  workers  from  looking  for  such  algorithms. 
Furthermore,  they  point  to  the  need  of  a  restricted  class  of  context 
free  languages  with  the  property  that  the  ambiguity  problems  concern¬ 
ing  this  class  is  decidable  and  this  class  of  languages  is  powerful 
enough  to  specify  the  "context-free  syntax"  of  the  programming  lan¬ 
guages.  The  class  of  deterministic  context  free  languages  [14, 17] 
seems  to  fit  these  requirements.  However,  further  research  is 
needed. 

8.4  Efficient  Parsing  Algorithms 

An  important  property  of  a  language  is  the  amount  of  time  neces¬ 
sary  to  recognize  a  sentence  of  the  language.  A  language  can  be  parsed 
in  linear  time  if  there  exists  an  algorithm  such  that,  for  an  arbitrary 
input  sentence  of  length  n  and  a  specification  of  the  language,  the  time 


*Let  G  =  (N,  E,  P,S)  be  a  grammar.  Then,  for  every  sentence  x  in 
L(G),  there  exist  c  derivation  S  ==g=>  =g=>  a2  ‘  ‘  ‘  ==g=->  an  x’ 
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(i.  e.  number  of  steps)  required  by  this  algorithm  to  parse  this  sen¬ 
tence  is  proportional  to  n.  The  general  context  free  grammars,  re¬ 
quire  more  than  linear  time  to  parse  a  context  free  language  in  gen¬ 
eral.  It  has  been  shown  by  Younger  [46]  that  there  exists  an  algo- 

3 

rithm  to  recognize  context  free  languages  in  time  n  . 

When  recognizers  are  really  built,  context  free  grammars  are 
rarely  used.  Restrictions  are  always  put  on  the  grammar  to  assure 
that  it  can  be  parsed  in  linear  time.  The  operator  precedence  gram¬ 
mar  defined  by  Floyd  [10]  is  the  most  well  known  one.  It  is  often  used 
to  recognize  portions  of  a  programming  language.  Unfortunately,  the 
operator  precedence  grammars  are  not  rich  enough  to  specify  all  the 
"context-free  syntax"  of  the  programming  languages.  Precedence 
grammars  [44],  extended  precedence  grammars  [28],  simple  deter¬ 
ministic  context  free  grammars  [22],  LR(k)  grammars  [21]  have  also 
been  defined.  All  of  them  specify  classes  of  languages  richer  than 
the  class  of  context  free  languages.  Theoretically  they  can  also  be 
parsed  in  linear  time.  However,  they  are  still  much  less  efficient 
than  the  operator  precedence  grammar  in  a  strictly  practical  sense. 

8.  5  Conclusion 

The  body  of  knowledge  in  the  area  of  formal  languages  has  been 
growing  very  rapidly  in  recent  years.  The  concept  of  a  grammar  has 
been  used  for  specifying  programming  languages  and  building  efficient 
compilers. 


132 

The  basic  drawback  in  the  definition  of  a  formal  language  is 
the  lack  of  meaning  associated  with  each  sentence.  Thus  it  is  not 
surprising  that  the  application  of  formal  languages  appears  to  be 
limited  to  the  syntax  specification  and  the  parsing  algorithms  and 
that  various  practical  problems  concerning  compilers  cannot  be 
answered  satisfactorily  by  the  theories  in  the  area  of  formal  lan¬ 
guages. 

The  next  step  requires  the  assignment  of  meaning  to  each 
sentence  in  a  formal  language.  Some  investigations  indicate  that 
the  complexity  of  the  research  problem  increases  considerably 
but  there  are  encouraging  results.  For  example,  it  is  known 
that  .  .~e  d-  es  ac  exist  an  algorithm  to  decide  whether  two  arbi¬ 
trary  context  free  grammars  generate  the  same  language  [24],  How¬ 
ever,  it  has  been  shown  [35]  that  there  exists  an  algorithm  to  decide 
whether  two  context  free  grammars  are  structurally  equivalent  in 
the  sense  that  they  not  only  generate  the  same  language  but  also 
"structure"  these  sentences  in  the  same  manner  (the  same  meaning). 

It  is  possible  that,  after  assigning  "meaning  ’  to  each  sentence  in  a 
language,  algorithms  to  solve  some  problems  which  have  been  proven 
to  be  unsolvable  at  least  this  seems  to  be  a  promising  direction  for 


future  research. 
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